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ABSTRACT 

The main parameters of state self-employment 
assistance (SEA) programs for unemployed workers were tested in two 
experimental demonstrations authorized by the North American Free 
Trade Agreement Implementation Act. The Washington State and 
Massachusetts Unemployment Insurance (Ul) Self-Employment 
Demonstrations evaluated the ability of the employment security and 
economic development systems to work together and help UI recipients 
create their own jobs by starting businesses. The classical 
experimental design randomly assigned applicants to either a 
treatment or a control group. Design differences between the two' 
demonstrations were a requirement in the Massachusetts authorizing 
legislation that eliminated those with a low predicted probability of 
exhausting UI benefits and eligibility of Washington participants to 
receive a lump-sum payment of remaining benefits. The demonstrations 
were initiated in six Washington and seven Massachusetts sites. 
Relatively few new UI claimants in both states chose to pursue 
self-employment when the opportunity arose. Findings from a follow-up 
survey approximately 31 months after assignment in Massachusetts and 
approximately 33 months in Washington, indicated that both 
demonstrations increased the likelihood of self-employment, increased 
the total time in employment, and reduced the length of unemployment. 
Both programs were cost effective from the nonpar t ic ipant and 
government perspective. (Appendixes include definitions of 
independent variables, siii-Tificant coefficients, and effects on 
subgroups.) (YLB) 
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ABSTRACT 



The North American Free Trade Agreement (NAFTA) Implementation Act (P.L. 103- 
182) authorizes states to establish self-employment assistance (SEA) programs for unemployed 
workers. As of the end of 1994, five states (California, Connecticut, Maine, New York and 
Rhode Island) have enacted enabling legislation to implement SEA programs for the unemployed. 

The main parameters of SEA programs were tested in two experimental demonstrations 
sponsored by the U.S. Department of Labor (DOL). The Washington State and Massachusetts 
Unemployment Insurance Self-Employment Demonstrations evaluated the ability of the U.S. 
employment security and economic development systems to work together and help 
Unemployment Insurance (UI) recipients create their own jobs by starting businesses. 
Preliminary results from these two demonstrations indicated that SEA is a viable reemployment 
option for some portion of the unemployed. Moreover, preliminary results suggested that the 
Massachusetts model, was likely to be a cost-effective approach for providing SEA to UI 
claimants. These early results were cited in the decision to authorize SEA for a five-year 
period. 

In this report, we present the fmal impact estimates of the Washington and Massachusetts 
UI Self-Employment Demonstrations. These fmal results largely reinforce the earlier 
preliminary fmdings and underscore the conclusion that SEA is a viable policy tool to promote 
the rapid reemployment of unemployed workers. The cumulative evidence from the preliminary 
and final evaluations suggests that SEA should be permanently incorporated into the U.S. 
employment security and economic development system. 
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EXECUTIVE SUMMARY 



The North American Free Trade Agreement (NAFTA) Implementation Act (P.L. 103- 
182) authorizes states to establish self-employment assistance (SEA) programs for unemployed 
workers. As of the end of 1994, five states (California, Connecticut, Maine, New York and 
Rhode Island) have enacted enabling legislation to implement SEA programs for the unemployed. 

The main parameters of SEA programs were tested in two experimental demonstrations 
sponsored by the U.S. Department of Labor (DOL). The Washington State and Massachusetts 
Unemployment Insurance Self-Employment Demonstrations evaluated the ability of the U.S. 
employment security and economic development systems to work together and help 
Unemployment Insurance (UI) recipients create their own jobs by starting businesses. 
Preliminary results from these two demonstrations indicated that SEA is a viable reemployment 
option for some portion of the unemployed. Moreover, preliminary results suggested that the 
Massachusetts model, was likely to be a cost-effective approach for providing SEA to UI 
claimants. These early results were cited in the decision to authorize SEA for a five-year 
' period. 

In this report, we present the final impact estimates of the Washington and Massachusetts 
UI Self-Employment Demonstrations. These final results largely reinforce the earlier 
preliminary findings and underscore the conclusion that SEA is a viable policy tool to promote 
the rapid reemployment of unemployed workers. The cumulative evidence from the preliminary 
and final evaluations suggests that SEA should be permanently incorporated into the U.S. 
employment security and economic development system. 
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The UI Self-Employment Demonstrations 

The programs discussed in this report ~ the Washington State Self-Employment and 
Enterprise Development (SEED) Project and the Massachusetts Enterprise Project ~ are the first 
two federally-sponsored self-einployment demonstration programs des ^ned to assist unemployed 
workers in the United States. Collectively, they are known as the Unemployment Insurance (UI) 
Self-Employment Demonstrations. Abt Associates Inc. and Battelle Memorial Institute designed 
and evaluated these experimental demonstrations. 

DemoQstration Design 

A classical experimental design was developed to rigorously evaluate program 
effectiveness. Applicants to the demonstration were randomly assigned either to a treatment 
group, which was eligible to receive all program services, or to a control group, which was not 
eligible to receive program services, but remained eligible for regular UI benefits. 

To promote early intervention, both the Massachusetts and the Washington projects 
targeted new UI claimants. Both excluded persons filing interstate claims, claimants who were 
employer-attached (i.e., on standby to return to their former employer), and claimants under 18 
years of age. 

While there were some minor differences in program design between the two 
demonstrations, a key design difference resulted from the authorizing legislation for the 
Massachusetts demonstration (Section 9152 of the Omnibus Budget Reconciliation Act of 1987). 
The legislation required that the demonstration program focus on UI claimants who were likely 
to exhaust their UI benefits To implement this legislative requirement, sample selection was 
based on a statistical model that predicted each new claimant's likelihood of UI benefit 
exhaustion. Those new claims with a low predicted probability of exhausting UI benefits were 
eliminated from the Massachusetts target group. 

A second major design difference was the financial assistance component in each of the 
demonstrations. In Massachusetts, participants were eligible to receive periodic self-employment 
assistance payments. In Washington, participants were eligible to receive similar periodic 
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payments. In addition, however, upon meeting certain milestones in the development of their 
businesses, Washington participants were eligible to receive their remaining available UI benefits 
in one lump-sum payment. 

Otherwise, the Washington and Massachusetts demonstration designs were very similar. 
Specifically, invited claimants were required to attend an orientation meeting and to submit an 
application to the program. Applicants who met ail the requirements were randomly assigned 
to either the treatment or the control group. Treatment group members were then offered 
business start-up services and financial assistance. 

Business Start-up Services and Financial Assistance 

The first step for Massachusetts participants was to attend an Enteiprise Seminar, a one- 
day training session conducted by one or more business experts. Within two weeks of the 
Enteiprise Seminar, participants were required to attend an individual counseling session with 
their business counselor. Massachusetts participants were also required to attend six workshop 
sessions on a variety of business topics. 

The financial assistance component in Massachusetts included payment of regular bi- 
weekly UI benefits, with an exemption from the regular UI work search requirements while in 
the demonstration. The demonstration design set the duration of the UI work search waiver at 
24 weeks. Thus, in Massachusetts, treatment group members could collect self-employment 
allowances through the 24th consecutive week of their UI claim. Since Massachusetts claimants 
are eligible for up to 30 weeks of UI benefits, after 24 weeks they were required to choose 
between continuing with their self-employment activities full-time or returning to UI for the 
remaining six weeks of UI eligibility and meeting the work search requirements. 

The business start-up services provided in the Washington demonstration differed in 
several respects from the Massachusetts demonstration. Within two weeks after random 
assignment, treatment group members were scheduled to attend a set of four business training 
sessions. The four Washington training sessions were presented during a one-week period; in 
contrast, the seven Massachusetts sessions were presented over a 12-week period. 



iii 



iu 



Executive Summary 



The financial assistance component of the Washington demonstration included both 
regular UI payments and eligibility for a lump-sum payment. That is, Washington treatment 
group members received regular bi-weekly UI payments while engaged in business start-up 
activities; in addition, they were eligible to receive a lump-sum payment equal to their remaining 
UI entitlement when they achieved five program milestones: 

• Completion of the training sessions; 

• Development of an acceptable business plan; 

• Establishment of a business bank account; 

• Satisfying all licensing requirements; and 

• Obtaining adequate financing. 

There was no lump-sum payment available to Massachusetts participants. 

Demonstration Implementation 

The Washington SEED Demonstration was initiated on a pilot basis in one site in 
September 1989, and was then implemented in five additional sites in February 1990. 
Demonstration intake activities continued through September 1990, with busmess support 
services available to demonstration participants through March 1991. In the six SEED 
demonstration sites, a total of 755 new claimants were offered demonstration services (i.e., 
assigned to the treatment group); a total of 752 new claimants who applied to SEED were 
assigned to the control group. 

The Massachusetts Enteiprise Project began operations in May 1990. Enrollment into 
the Enteiprise Project took place in three distinct phases: the first enrollment phase took place 
in 1990 (May - September), the second in 1991 (April - October), and the third in 1992-93 
(March 92 - April 93). Over all three phases of operation, a total of 614 new UI claimants were 
enrolled in the Enteiprise Project in the seven sites; an additional 608 new UI claimants were 
assigned to the control group. 

In both Washington and Massachusetts, only a relatively small fraction of targeted UI 
claimants met the initial demonstration requirements of attending an orientation meeting and 
submitting an application. In Washington, four percent of targeted UI claimants completed the 
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initial requirements and were eligible for SEED participation; in Massachusetts, an even smaller 
proportion, two percent, met the same requirements and were eligible for Enterprise Project 
participation. Thus, relatively few new UI claimants choose to pursue self-employment when 
the opportunity arises. 

The main findings from our analysis of program implementation are: 

• In Washington, 7.5 percent of the 42,350 targeted new UI claimants invited to a 
meeting about the SEED Demonstration attended that meeting. In Massachusetts, 
4.2 percent of the 63,921 invitees attended a meeting about the Enterprise 
Project. 

• In both demonstrations, the recruitment and intake procedures were implemented 
as designed, meeting the program objective of early inter\'ention. In Washington, 
attendance at the information session occurred on average 18 days after the initial 
UI claim date; in Massachusetts this interval was on average 33 days. The longer 
interval in Massachusetts relative to Washington is largely due to the fact that 
Washington was a wage-reporting state for UI purposes and Massachusetts was 
a wage-request state (i.e., had to request wage data from employers to determine 
claimants' eligibility for UI benefits) during the demonstration. Massachusetts 
procedures have recently changed and both are now wage-reporting states. 

• In Washington, the 1,507 claimants who applied and were randomly assigned 
represent 3.6 percent of the targeted UI claimants. In Massachusetts, the 1,222 
applicants who were randomly assigned represent 1.9 percent of the targeted UI 
claimants. 

• In Washington, treatment group members began training services, on average, 
within 6 weeks of their initial UI claim date. In Massachusetts, training services 
began within 9 weeks. 

• In both states, a high proportion of treatment group members participated in 
program services such as business training and counseling. In Washington, for 
example, approximately 85 percent of the treatment group attended the first 
training session and 70 percent received some counseling. In Massachusetts, the 
proportions were even higher. 

• Program design differences led to greater utilization of counseling services in 
Massachusetts than in Washington. The mean hours of counseling in 
Massachusetts was 7.5 hours per participant, while the mean number of hours in 
Washington was 1.5. 
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• In both states, treatment group members received regular benefit payments while 
developing their businesses. Additionally, in Washington, treatment group 
members were eligible for a lump-sum payment (equal to their remaining 
available UI benefits) after achieving five program milestones. A total of 451 
treatment group members (60 percent) received a lump-sum payment averaging 
$4,225. There was no lump-sum payment in Massachusetts. 

Data Sources 

Both the Washington and Massachusetts analyses are largely based on data from the 
Participant Tracking System (PTS), an on-line database system developed by DOL to provide 
information about project participants and project services, and two telephone surveys. The first 
Washington telephone followup survey was conducted, on average, 21 months after random 
assignment; a total of 1,204 sample members (or 80%) responded to this survey. In 
Massachusetts, the first telephone followup survey was conducted, on average, 19 months after 
random assignment; 449 sample members (or 80%) responded to this survey. The second 
telephone followup survey in each State was conducted approximately one year after the first 
survey, and approximately 90% of the Wave I respondents, responded in each State. In 
Washington, the second followup survey was conducted, on average, 33 months after random 
assignment; in Massachusetts, 31 months after random assignment. 

Demonstration Results 

In the fiill report we present impact results for two observation periods: the period from 
random assignment to the first followup survey, and the period from random assignment to the 
second followup survey. For simplicity, the text below summarizes only the results for the 
longer observation period (approximately 31 months in Massachusetts and 33 months in 
Washington). 

Self-Employment Impacts. The Washington SEED program had a very large and 
positive impact on the self-employment experiences of UI claimants. Specifically, treatment 
group members were estimated to be much more likely than controls to have a self-employment 
experience, to spend more time per year in self-employment, and were more likely to be still 
self-employed at the time of the second followup survey. Consistent with these results, we 
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found that the SEED program increased claimants' self-employment earnings by over $1 ,600 per 
year. 

In contrast, The Massachusetts Enterprise Project had significant positive impacts on only 
some of the self-employment outcomes analyzed. Specifically, treatment group members were 
more likely than controls to have a self-employment experience and to spend more time per year 
in self-employment. In contrast to the Washington demonstration, treatment group members 
were not significantly more likely than controls to be self-employed at the time of the second 
survey; furthermore, the Massachusetts demonstration did not have a significant impact on self- 
employment earnings. 



Self-Employment Impacts 





Washington 


Massachusetts 


Percent Self-Employed 
Since Random Assignment 


+22%'" 


-1-12%" 


Time Self-Employed 
Per Year 


+2.0"' 
months 


4-0.8* 
months 


Percent Self-Employed at 
Second Survey 


+ 12%"' 


4-5% 


Self-Employment Earnings 
Per Year 


+$1,615" 


4-$l,219 



♦♦♦Indicates coefficient is significantly different from zero at the .01 level. 
♦♦Indicates coefficient is significantly different from zero at the .05 level. 
♦Indicates coefficient is significantly different from zero at the .10 level. 



Wage and Salary Outcomes. In Washington, the estimated program impacts on wage 
and salary employment measures were consistently negative. Specifically, SEED significantly 
reduced claimants' likelihood of working in wage and salary employment, reduced number of 
months per year working in wage and salary employment, and reduced earnings from wage and 
salary employment by about $1,800 per year. 
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In contrast to the Washington results, the Massachusetts estimated impacts on wage and 
salary outcomes were generally insignificant; only the impact on earnings was significant. 
Indeed, the Enterprise Project significantly increased claimants' earnings from wage and salary 
employment by over $3,000 per year. This positive earnings impact contrasts dramatically with 
the negative Washington impact on earnings from wage and salary employment. 



Wage and Salary Impacts 





Washington 


Massachusetts 


Percent Employed 

Since Random Assignment 


-6%"' 


-4% 


Time Employed 
Per Year 


-O.T" 
months 


+0.(5 
months 


Percent Employed at 
Second Survey 


-3%' 


+ 1% 


Wage & Salary Earnings 
Per Year 


-$1,780" 


+$3,053" 



***Indicates coefficient is significantly different from zero at the .01 level. 
**Indicates coefficient is significantly different from zero at the .05 level. 
♦Indicates coefficient is significantly different from zero at the .10 level. 



Combined Self-Employment and Wage and Salary Outcomes. The Washington 
demonstration did not significantly increase the combined likelihood of employment in either a 
wage and salary job or in self-employment. The program did, however, increase the time 
worked per year and increased the likelihood of being employed at the time of the second 
survey. On the other hand, the impacts on total earnings (self-employment and wage and salary 
earnings) were not statistically different from zero. Thus, it appears that SEED'S positive 
impact on self-employment earnings together with its negative impact on wage and salary 
earnings resulted in a zero impact on claimants' total earnings. 

In contrast, the effect of the Massachusetts Enterprise Project on total employment and 
earnings outcomes was large and consistent. Specifically, the program significantly increased 
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the likelihood of finding employment, increased the time worked per year, and increased the 
likelihood of being employed at the time of the second survey. Perhaps the greatest impact, 
however, was on total earnings. The demonstration increased combined annual earnings by 
nearly $6,000. Thus, it appears that the Enterprise Project had a dramatic positive impact on 
claimants* total earnings. 



Combined Self-Employment and Wage and Salary Impacts 





Washiagton 


Massachusetts 


Percent Employed 

Since Random Assignment 


+0% 


+5%" 


Time in Employment 
Since Random Assignment 


+ 1.1'" 

months 


+ 1.9'" 

months 


Percent Employed at 
Second Survey 


+6%" 


+6%' 


Total Earnings 

Since Random Assignment 


+$289 


+$5,940'" 



***Indicates coefficient is significantly different from zero at the .01 level. 
♦♦Indicates coefficient is significantly different from zero at the .05 level. 
♦Indicates coefficient is significantly different from zero at the .10 level. 



Nonparticipant Employment Outcomes. The Washington demonstration had a 
significant positive impact on the employment of nonparticipants (family and nonfamily 
employees, excludmg the busmess owner(s)) during the 33-month observation period. 
Specifically, we estimate that SEED businesses created a total of 316 nonparticipant jobs; control 
group businesses, created 128 jobs. The difference of 188 nonparticipant jobs may be attributed 
to the Washington demonstration. In contrast, the Massachusetts demonstration did not have 
a significant impact on nonparticipant job creation. 

Benefit Payment Outcomes. The Washington demonstration significantly increased 
receipt of total benefits. Taking into account both the regular UI benefits and the lump-sum 
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payment provided to treatment group members who met the required milestones for starting a 
business, SEED increased total benefits by approximately $1,000. In contrast to the Washington 
results, the Massachusetts demonstration significantly reduced receipt of total benefits by nearly 
$900. 

Cost-Benefit Analysis 

The Massachusetts Enterprise Project generated large net benefits from each of several 
perspectives examined. From the perspective of participants, the net benefits during the 31- 
month observation period was over $11,000; from nonparticipants' perspective, the net benefit 
was over $2,000; and, from society's perspective, the net benefit was over $13,000. An 
examination of the benefits and costs from government's perspective also indicated that the 
Massachusetts Enterprise Project had large (over $2,000) net benefits. Given these strong 
results, the Massachusetts Enterprise Project appears to be a highly cost-effective policy tool for 
assisting UI claimants who are interested in pursuing self-employment. 

The Washington SEED program had positive net benefits from participants' perspective 
(approximately $2,000) and from society's perspective (approximately $700). From the 
nonparticipant (i.e., taxpayer) and government perspectives, however, there were net costs of 
over $1,200. Given these results, the Washington program, like the Massachusetts program, 
is cost-effective from participants' and from society's perspectives. However, in contrast to the 
Massachusetts progran. , the Washington program has significant redistributional implications and 
net costs to the government. 

Conclusions 

The results of this study indicate that both the Washington and Massachusetts 
demonstrations increased the likelihood of self-employment, both increased the total time in 
employment, and both reduced the length of unemployment. Moreover, both programs are cost- 
effective from society's perspective; however, only the Massachusetts program is cost-effective 
from the nonparticipant and government perspectives. These results indicate that SEA is a cost- 
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effective approach to promote the rapid reemployment of unemployed workers and should be 
pemanently incorporated into the U.S. employment security and economic development system. 
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On December 8, 1993, President Clinton signed the North American Free Trade 
Agreement (NAFTA) Implementation Act. (P.L. 103-182). Section 507 of the Act authorizes 
states to establish self-employment assistance (SEA) programs as part of the state unemployment 
compensation system. To establish SEA programs, states must enact legislation to conform to 
the Federal legislation. As of December 1994, five states have enacted such legislation: 
California, Connecticut, Maine, New York, and Rhode Island. 

. The provisions of the Act give states the ability to add self-employment programs to the 
current mix of reemployment policy tools available to help unemployed workers. Specifically, 
the Act allows states to pay a self-employment allowance in lieu of unemployment compensation 
to unemployed workers while they are establishing businesses and becoming self-employed. To 
qualify for these self-employment allowance payments, program participants must receive 
business counseling, attend entrepreneurial training courses, and receive other technical 
assistance services in support of their self-employment effort. 

The main parameters of this new self-employment assistance program are based on two 
experimental dei onstrations sponsored by the U.S. Department of Labor (DOL). Tlie 
Washington State and Massachusetts Unemployment Insurance Self-Employment Demonstrations 
were initiated by DOL to test the ability of the U.S. employment security and economic 
development systems to help Unemployment Insurance (UI) recipients start businesses. These 
demonstrations provided selected UI claimants with business development assistance, in the form 
cf entrepreneurial training, business support services and financial assistance. 

Similar self-employment programs for the unemployed have been implemented in a 
number of Western European countries, including Belgium, Denmark, Finland, France, 
Germany, Ireland, Italy, Luxembourg, the Netheriands, Norway, Portugal, Spain, and Sweden. 
The two largest programs are in France and Great Britain. The French Chomeurs Createures 
("Unemployed Entrepreneurs") provides participants with a single lump-sum payment for 



ERIC 2o 



Chapter 1 • Introduaion 



business start-up capital. The British Enterprise Allowance Scheme provides participants with 
a weekly self-employment allowance in lieu of their regular unemployment benefits. In a 
number of respects, the Washington Demonstration resembles the French program; similarly, 
the Massachusetts Demonstration resembles the British program. Thus, the UI Self-Employment 
Demonstrations test two alternative models of self-employment assistance programs. 

Background of the UI Self-Employment Demonstrations 

Traditionally, the unemployment compensation system has provided income support for 
workers who are temporarily laid off or expect to be unemployed for only a short time. In 
recent years, however, an increasing number of workers have permanently lost their jobs. These 
displaced workers require more than temporary income maintenance to re-enter the workforce. 
For some, self-employment may be the best path to re-enter the workforce. 

Over the past decade, the U.S. Department of Labor (DOL) has launched a series of 
experimental demonstrations to investigate alternative reemployment strategies. The strategies 
tested in these demonstrations have included job search assistance, occupational retraining, 
relocation assistance, and reemployment bonuses (monetary incentives for early re-employment 
in new jobs). In addition to these demonstrations that focused on reemployment into wage and 
salary jobs, DOL also initiated experimental demonstrations to test the efficacy of self- 
employment assistance as a reemployment strategy. 

In the late 1980's, DOL initiated the Washington State and Massachusetts UI Self- 
Employment Demonstrations to test the ability of the employment security and economic 
development systems to help UI recipients start businesses. Both of these demonstrations 
provided participants with business development assistance, in the form of entrepreneurial 
training, business support services, and fmancial assistance. A brief description of each of the 
demonstrations follows. 
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Washington Self-Employment and Enterprise Development (SEED) Demonstration 

The first federally- sponsored project in the U.S. to test the use of self-employment 
programs as a reemployment strategy was the Washington Self-Employment and Enterprise 
Development (SEED) Demonstration. The SEED Demonstration was initiated on a pilot basis 
in one site beginning in September 1989 and was then implemented in five additional sites in 
February 1990. Demonstration intake activities continued through early September 1990, with 
business support services available to demonstration participants through March 1991. 

To allow rigorous evaluation of program effectiveness, the SEED Demonstration used 
a classical experimental design with random assignment of eligible claimants interested in 
starting their own businesses. These Individuals were randomly assigned to either a treatment 
group, that was eligible to receive all program services, or to a control group, that was not 
eligible to receive program services, but remained eligible for regular UI benefits. Using this 
experimental design, the impacts of program services can be measured directly by the difference 
in outcomes between the treatment and control groups. A total of 755 new claimants were 
enrolled in SEED in the six sites and offered demonstration services; 752 new claimants who 
applied to SEED were assigned to the control group. 

The Massachusetts UI Self-Employment Demonstration (The Enterprise Project) 

A second self-employment demonstration was mandated by Section 9152 of the Omnibus 
Budget Reconciliation Act (OBRA) of 1987. Tltis Act authorized up to three States to participate 
in a self-employment expeiimental demonstration. In 1988; Massachusetts agreed to participate 
in the demonstration. In designing the Massachusetts UI Self-Employment Demonstration 
(Enterprise Project), the Department of Labor, the State of Massachusetts, and the res archers 
(Abt Associates/Battelle) were guided by two objectives. The first was to develop a program, 
consistent with the authorizing legislation, designed to facilitate self-employment for UI 
claimants who choose this avenue. The second objective was to develop a design that would 
pennit a scientifically valid program evaluation. 
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The provisions of the authorizing legislation mandated a number of important 
demonstration design features. For example, the Act required that the demonstration target self- 
employment services to UI claimants "likely to receive regular or extended benefits for the 
maximum number of weeks that such compensation is made available under the State law during 
such benefit year" (Section 9152(i)). Another important provision of the legislation was to 
require participating states to reimburse the Unemployment Trust Fund for any excess costs 
incurred as a result of the demonstration (Section 9152(c)). Excess costs arise when 
demonstration treatment group members, on average, collect more self-employment allowances 
than the amount of Ul benefits they would have collected in the absence of the demonstration 
(as measured by the experience of control group members). Finally, the authorizing legislation 
required the demonstration to be implemented over a three- year period. 

The Massachusetts Enterprise Project began operations in May 1990. Enrollment into 
the Enterprise Project took place during three distinct phases. The first enrollment phase took 
place in 1990 (May - September), the second in 1991 (April - October), and the third in 1992-93 
(March 92 - April 93). As in the Washington Demonstration, the Massachusetts Demonstration 
used a classical experimental design with random assignment of eligible claimants interested in 
starting their own businesses. A total of 1 ,222 UI claimants were randomly assigned to either 
the treatment group (614) or the control group (608) during all three enrollment periods.' 

Organization of the Report 

In this report we present the final impact results and the cost-effectiveness analysis of the 
Washington and Massachusetts Demonstrations. Previous reports have described the 
implementation and early impacts of the two demonstrations. Specifically, a complete 
description of the SEED Demonstration implementation, is found in Johnson and Leonard 
(1991); the early implementation of the Enterprise Project is described in Benus et al. (1990); 
the first impact results of the SEED Demonstration are described in Benus, Johnson et al. 



' In 1990. 207 UI claimants were randomly assigned; in 1991, 314 were randomly assigned, and in 1992-93, 
701 were randomly assigned. 
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(1993); finally, a comparison of the early impacts of the SEED Demonstration and the early 
impacts of the Enterprise Project is presented in Benus et al. (1994). 

In this report, we present the final impact results of the UI Self-Employment 
Demonstrations. These impact results are based on administrative data collected throughout the 
thre2-year demonstration observation period as well as on survey data collected in two follow-up 
surveys. The first survey was conducted approximately 20 months after random assignment and 
the second survey was conducted approximately 33 months after random assignment. 

This report also presents, for the first time, information on the cost-effectiveness of the 
two self-employment assistance programs that were tested as part of the UI Self-Employment 
Demonstrations. The cost-effectiveness results are based on a comparison of program impacts 
and program costs. For program impacts, we use the 33-month impact estimates derived in 
Chapters 7 and 8. Program costs are derived from data provided by program operators. By 
comparing program benefits with program costs, we can evaluate if the programs tested are cost- 
effective. 

This report is organized in three mr in parts. Part I (Chapter 1 through Chapter 6) gives 
an overview of the UI Self-Employment Demonstrations, the implementation experiences, and 
the data used in the analysis. 

The details of the experimental and operational design of the two demonstrations are 
presented in Chapter 2. For each demonstration, we describe the design of the targeting criteria, 
the recruitment and intake process, the application and random assignment procedures, the 
program services, and the financial assistance provided. 

In Chapter 3, we describe the implementation of the Washington Demonstration. Since 
the implementation of SEED has been described in detail in Johnson and Leonard (1991), we 
only present a brief summary of the SEED implementation experiences. We summarize 
demonstration intake results, business support services, and fina. :ial assistance provided to 
SEED participants. 

In Chapter 4, we describe the implementation of the Massachusetts Enterprise Project in 
greater detail. We first note demonstration intake results and the characteristics of Enterprise 
Project participants. Next, we look at the main changes in program implementation procedures. 
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We also analyze participation in demonstration ..activities, the timing of these activities, and the 
changes in UI regulations that were implemented during the course of the demonstration. 

In Chapter 5, we compare the main implementation results of the Washington and 
Massachusetts Demonstrations. By comparing program experiences, we gain insights about the 
impact differences that are found in later chapters. 

Chapter 6, the last chapter in Part I, provides a description of the data sources used in 
the impact and cost-effectiveness analysis. We first describe the Participant Tracking System 
(PTS), an on-line database system developed by DOL that provides data on personal 
characteristics, demonstration services, business information, and UI benefits information. We 
then describe the administrative records data, including the State UI Wage Records. Finally, 
we describe the contents of the followup survey and the procedures used in its administration. 

Part n of the report (Chapter 7 through Chapter 9) provides our estimates of the program 
impacts on key outcome measures. Chapter 7 focuses on the Washington Demonstration, 
Chapter 8 focuses on the Massachusetts Demonstration, and Chapter 9 compares the impact 
estimates from the two demonstrations. 

We begin Chapter 7 by describing the characteristics of Washington participants who 
entered self -employment during the observation period. Following this descriptive analysis, we 
evaluate SEED program impacts on the likelihood of being self-employed, time in self- 
employment, and earnings from self-employment. We also investigate other demonstration 
impacts, including the unpacts on UI outcomes. 

In Chapter 8, we present an analysis of the impacts of the Massachusetts Enterprise 
Project. As in the SEED analysis, we investigate self-employment outcomes, wage and salary 
outcomes, combined (self-employment or wage and salary) outcomes, and other outcomes. 

In Chapter 9, we compare the demonstrations' impacts. We also investigate the potential 
impact of "profiling" (i.e., targeting claimants who are likely to exhaust benefits) on the 
observed impact results. This latter issue is of particular interest to policymakers since States 
must use profiling in implementing their self-employment assistance programs. 

Part ni, which includes only Chapter 10, presents cur analysis of the cost-effectiveness 
of the two demonstrations. 
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In this chapter we compare and contrast the experimental and operational design features 
of the Washington and Massachusetts demonstrations. We focus on the features of the 
demonstrations that are important for understanding and interpreting the impact results that will 
be presented later in this report. 

As an aid for understanding the two experimental designs, we present in Exhibit 2. 1 a 
flow chart depicting the intake and random assignment procedures used in each of the 
demonstrations. As indicated by the exhibit, the overall flow is similar in both demonstrations. 
The main differences in experimental design are discussed in depth in the following subsections. 

Targeting Demonstration Participants 

Both the Massachusetts and the Washington projects targeted new UI claimants. In both 
states, the following claimants were excluded: 

• Persons filing interstate claims; 

• Claimants who were employer-attached (i.e., on standby); and, 

• Claimants under 18 yean of age. 

In addition, the Washington demonstration excluded persons who were full-referral union 
members and those filing claims backdated more than 14 days; the Massachusetts demonstration 
excluded claimants eligible for less than 26 weeks of UI benefits. 

Since the authorizing legislation for the Massachusetts demonstration required that the 
program focus on UI claimants who were likely to exhaust their UI benefits, further targeting 
was necessary to select those claimants likely to exhaust benefits. To implement this legislative 
requirement, the sample selection was based on an algorithm that predicted each claimant's 
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likelihood of UI benefit exhaustion.^ Using this algorithm, a numerical probability of 
exhaustion was calculated for each new Massachusetts claimant in the target population. Entry 
into the demonstration was then restricted to those with a high predicted probability of 
exhausting UI benefits.^ 

Recrutting Participants 

The next step in both demonstrations was to recruit claimants interested in self- 
employment. Once identified, eligible claimants were sent a letter inviting them to attend an 
initial information session. To ensure that the most highly-motivated claimants were identified 
for the self-employment program, strict time limits were established for attending this session. 
Only claimants who met these time constraints were permitted to continue in the demonstration. 
In this way, self-screening eliminated the less-motivated claimants from the demonstration. 

In both demonstrations, the information sessions were held in the local UI office in which 
the claimant filed his or her claim.* The key difference between the Washington and 
Massachusetts information sessions was in the format of the presentations. In Washington, a 
local UI office staff person took attendance, introduced a set of two videos (covering the key 
features of SEED and the risks and rewards of self-employment), showed the videos, and 
answered questions at the end of the session. In Massachusetts, on the other hand, the 
presentations were given by a local UI office staff member and a business development expert. 
The local UI staff member described the demonstration procedures and distributed applications 
at the end of the session; the business development experts gave a presentation on the risks and 
rewards of self-employment. 



' For details on the algorithm, see Benus, et al. "Massachusetts UI Self- Employment Demonstration Interim 
Report to Congress" in Self-Employment Programs for Unemployed Workers, U.S. Department of Labor, 
Employment and Training Administration, 1992. 

' In addition, to regulate the flow of claimants into both demonstrations, further subsampling occurred in the 
largest demonstration sites. 

" In Massachusetts, the schedule for the sessions was weekly during year one. The schedule was modified after 
the first year and information sessions were conducted on a bi-weekly basis in the three sites with the lowest number 
of new UI claimants. In Washington, information sessions were conducted bi-weekly (cvcry-othcr Friday) except 
during the pilot in Vancouver when the sessions were held weekly. 
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The purpose of the initial information session was to provide claimants with sufficient 
information about the self-employment program to decide whether or not to apply for program 
services. During the session, claimants were provided with basic information about the risks 
and rewards of self-employment and the key features of the demonstration. The Massachusetts 
sessions lasted approximately 60 minutes; the Washington sessions were scheduled to last 
approximately 30 to 45 minutes. 

Application and Random Assignment 

At the conclusion of the initial information sessions, claimants who were still interested 
in applying to the program took application materials home to complete. Applicants were 
required to return the completed materials within seven days. The completed applications 
contained personal background information and a description of the applicant's proposed business 
idea. 

The applications were reviewed by project staff for timeliness and completeness. The 
business ideas were also reviewed to ensure that they conformed to established project 
guidelines.' The business idea was not evaluated on its merit or on its likelihood of success. 
Those applicants who submitted the applications on time and satisfied the project guidelines were 
eligible for random assignment. Eligible applicants were then randomly assigned to either the 
treatment group that was eligible to receive business development services and financial 
assistance or to a control group that was not. Control group members were informed that they 
were not selected for the experimental program and that they must continue to meet all UI 
eligibility requirements. Both treatment and control group members were informed that they 
would be contacted in the future for research purposes. 



' For example, the business idea must be legal and the participant must have day-to-day control of the business. 
It is important to note that the business idea itself was not evaluated on its likelihood of success. In other words, 
wc did not attempt to identify "winners" for inclusion in the demonstration. 
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Business Start-up Services and Financial Assistance 

The flow chart in Exhibit 2.2 highlights the differences in business start-up services in 
the two demonstrations. We summarize these below. 

Massachusetts 

Once they were assigned to the treatment group, the first step for Massachusetts 
participants was to attend an Enterprise Seminar, a one-day session that was conducted by one 
or more business experts.' This intensive training session followed a standard curriculum and 
covered topics such as developing a business mindset, business organizational structures, 
marketing, business plan development, personnel issues, and business management. Within two 
weeks of the Enteiprise Seminar, participants were required to attend an individual counseling 
session with their business counselor. This counseling session lasted approximately one hour. 
In addition to this required counseling session, participants were encouraged to schedule 
additional counseling sessions with their business counselors as needed. Massachusetts 
participants were also required to attend Enterprise Workshops, six two-hour group sessions 
focused on the following topics:' 

• Marketing; 

• Personal effectiveness and selling; 

• Cash flow; 

• Financing; 

• Legal requirements and insurance; and, 

• Bookkeeping/ taxes. 

These six sessions were offered over approximately a twelve-week period during the first and 
second years. During the third year, the schedule for the sessions was streamlined, so that all 



* In 1990, this session was eight hours long. Program adniinistrators determined that eight hours was too long 
and difficult for participants. As a result, in 1991, the session was shortened to 4'^ hours (in 1992 it was 
lengthened to 5 'A hours). 

' During 1990, the sessions were called Bi-Weekly Meetings and did not follow a structured set of topics. In 
1991 , a more structured schedule of topics was developed for these sessions. In 1992, the sessions were extended 
to 2 1/2 hours in length and participants were required to give l5-minute presentations of their business plans. 
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sessions could be completed over a nine week period. During this period, participants were 
encouraged to develop a business plan with the assistance of their counselors. 

The financial assistance component in Massachusetts included payment of regular bi- 
weekly UI benefits, with an exemption from the regular UI work search requirements while in 
the demonstration. The demonstration design set the duration of the UI work search waiver at 
24 weeks. Thus, in Massachusetts, treatment group members could collect self-employment 
allowances through the 24th consecutive week of their UI claim. Since Massachusetts claimants 
were eligible for up to 30 weeks of UI benefits, at the 24th week they were required to choose 
between continuing with their self-employment activities full-time or returning to UI for the 
remaining six weeks of UI eligibility and meeting the work search requirements.* 

To provide additional financial support, the Enterprise Project developed a loan program 
through ShawmutBank, a large regional bank with branches in each of the demonstration sites. 
As part of this program, participants' loan applications were given consideration, even if the size 
of the loan fell below normal minimum levels. 

Washington 

The business start-up services provided in the Washington demonstration differed from 
the services provided in the Massachusetts demonstration. Within two weeks of random 
assignment, treatment group members were scheduled to attend a set of four business training 
modules covering the following topics: 

• Business feasibility; 

• Marketing; 

• Finance and accounting; and, 

• Organization and management. 



' During the third year of implementation (1992-1993), slightly different rules were in effect regarding UI 
eligibility. During this period, when Extended Benefits were available, Massachusetts UI claimants were eligible 
for only 26 weeks of regular benefits. That is, Massachusetts passed a law which stated that all claimants filing 
during the time when Extended Benefits were in affect would be eligible for 26 weeks of regular UI benefits and 
then would become eligible for Federal Extended Benefits. This change in law did not affect the total amount of 
benefits claimants could receive, but did affect the amount of benefits paid from the Massachusetts UI Trust Fund. 



13 



r "1 



Chapter 2 • Experimental and Operational Designs 



It took approximately 20 hours of classroom time to cover these four topics; the Massachusetts 
classroom sessions (i.e., the Enterprise Seminar and six Enterprise Workshops) took 
approximately the same classroom time to complete. The four Washington training modules 
were presented m four sessions during a one-week period; in contrast, the seven Massachusetts 
sessions were j^resented over a 12-week period. 

The traiiung modules introduced claimants to the need for developing a comprehensive 
business plan. Individualized business plans were then developed by participants with the 
assistance of a business development specialist. Additional assistance in developing a business 
plan was offered through Entrepreneur Club meetings, which were scheduled monthly. These 
meetings provided participants with peer support and advice throughout their demonstration 
participation. 

The financial assistance component of the Washington demonstration included regular bi- 
weekly UI payments while participants were engaged in business start-up activities as well as 
eligibility for a lump-sum payment of the remaining available UI benefits when they completed 
the following five milestones: 

• Completion of the training sessions; 

• Development of an acceptable business plan; 

• Establishment of a business bank account; 

• Satisfaction of all licensing requirements; and, 

• Obtaining adequate financing. 

Following the completion of these milestones, participants were eligible for a lump-sum payment 
equal to their remaining UI entitlement at the time. Because the remaining entitlement at any 
point in the claim is the maximum benefits payable less the amount of UI benefits already paid 
out, the amount of the lump-sum payment depended on the participant's UI entitlement, as well 
as the time taken to achieve the milestones. 

Although the lump-sum payment component of the SEED Demonstration was intended 
to simulate a cash-out of UI benefits, it was not strictly possible to test a cash-out policy because 
UI is an entitlement program that could not be denied for demonstration purposes. 
Operationally, this meant that participants could return to the regular UI program after receiving 
their lump-sum payment and draw the remainder of their UI entitlement in the form of bi-weekly 
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payments provided they met the normal UI eligibility requirements, including the work search 
requirement.' Nonetheless, most claimants thought it was a cash-out of their UI benefits and 
very few returned to UI. 

One key element in the Washington demonstration design was the role played by the 
business development specialists. These specialists provided ongoing counseling, assisting each 
participant in his/her pursuit of the five program milestones that were required to receive a 
lump-sum payment. The business development specialists were also responsible for conducting 
a "milestone review" to determine if all milestones had been attained. After the business start- 
up, the business specialists' responsibilities included technical assistance on an as-needed basis 
and a business status review, conducted approximately two months following receipt of the 
lump-sum payment. 

Participation Requirements 

. Each demonstration required participants to complete certain activities within a specified 
timeframe. Other activities were considered optional. In Washington, participants were 
required to attend the four training modules (or ask to have them waived). Counseling sessions 
were available to participants who wanted them, but were not required. Entrepreneur Club 
meetings were an optional activity. 

In Massachusetts, participants were required to attend the Enterprise seminar, one 
counseling session, and six Enterprise workshops. Additional counseling sessions were optional. 



' Because the lump-sum payments were paid out of Federal research funds -- not State UI funds ~ they did 
not affect a participant's UI net balance available. 
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DEMONSTRATION IMPLEMENTATION 



In this chapter, we review the Implementation of the Washington Self-Employment and 
Enterprise Development (SEED) Demonstration. A more complete description of the SEED 
demonstration implementation may be found in Johnson and Leonard (1991). Here, we present 
only those attributes of the implementation that are important for understanding the impact 
analysis that wiU be presented in later chapters. 

In the following sections, we provide a brief overview of demonstration implementation 
experiences. We first describe the flow of claimants through the demonstration intake process 
— from the identification of targeted claimants through random assignment. This includes 
evidence on the comparability of the treatment and control groups, as well as the timing of 
intake activities. We then describe the business support services and financial assistance 
received by treatment group members from the SEED Demonstration. 

Demonstration Intake 

The SEED Demonstration was implemented on a pilot basis in one site (Vancouver) in 
September 1989, and in five additional sites beginning in February 1990.'° Demonstration 
intake activities continued through September 1990, with business support services available to 
demonstration participants through March 1991. 

Development of Experimental Sample 

As described in Chapter 2, the development of an appropriate experimental sample of 
SEED Demonstration participants involved several steps. The first step was to identify targeted 

Because only minor changes vere made prior to full implementation, we combine the daia for the relatively 
few claimants who were part of the pilot study with the daU for the full demonstration sample. 
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new claimants without immediate job prospects and invite those interested in reT-employment 
to attend an initial information session, called Awareness Day in the SEED Demonstration. 
A total of 42,350 invitation letters were sent to targeted claimants in the six sites during the 
intake period." The six sites in the demonstration were: Vancouver, Olympia, King County, 
Snohomish County, Wenatchee, and Yakima. The location of these demonstration sites is 
depicted in Exhibit 3.1. 

In Table 3.1, we provide information on the characteristics of new TJI claimants in the 
six demonstration sites. Additionally, the characteristics of targeted claimants are compared with 
the characteristics of those who were excluded from the target group and not invited to 
participate. These results are based on Participant Tracking System (PTS) data'^ and 
Unemployment Insurance (UI) wage records for a 10 percent random sample of new claimants 
who filed for UI benefits in the six sites during the demonstration intake period. As Table 3. 1 
indicates, about 62 percent of all new claimants in the six sites were male, 84 percent were 
white, and 30 percent attended or completed college. The mean age was 36 years, with 22 
percent being at least age 45 or older. The mean earnings in covered employment during the 
four calendar quarters prior to the quarter of filing for benefits was just under $15,000. The 
mean weekly UI benefit amount (WBA) for all new claimants was $152, with a mean maximum 
benefits payable of $4,002. 

The results in Table 3.1 cleaily indicate that targeted claimants were quite different from 
non-targeted claimants in ways that could be expected given the targeting criteria. In particular, 
non-targeted claimants were much more likely to be union members and on standby. As a 
result, non-targeted claimants were much less likely to be in professional, technical, or 
managerial occupations or in clerical occupations. They were more likely than targeted 
claimants to be male, white, and slightly older, and were less likely to have any post-secondary 
education. In addition, non-targeted claimants had considerably higher earnings in the prior 
year, and correspondingly higher average weekly benefit amount and maximum benefits payable. 



" The two primary reasons that claimants were excluded from the SEED target population were: (1) employer 
attachment (50.3 percent of those excluded were on standby) and, (2) the claimant was a member of a ftill-referral 
union (28.9 percent). Only 18.0 percent were excluded because they had a backdated claim and very few claimants 
were excluded from the SEED target group because they were under age 18 or had filed an interstate claim. 

The participant tracking system (PTS) and its data are described in Chapter 6. 
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Table 3.1 
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Claimant Characterisdcs 


0ilvited to 


(Not Invited to 


All New 




Awareness vscy) ■. 


Awareness my} 


VUUlTlHlllS 


D«nographics 








Percent male 


57% 


80% 


62% 


Percent white 


82% 


91% 


OA of 

84% 


Percent high school graduate 


44% 


56% 


47% 


Percent some college 


22% 


19% 


1 of 

21 % 


Percent college graduate 


10% 


4% 


9% 


Mean education (in years) 


12% 


12% 


12% 


Percent age <, 24 


18% 


12% 


17% 


Percent age >. 45 


21% 


25% 


22% 


Mean age (in years) 


35 


37 


36 


Prior Work Experience 








Percent union hiring hall member 


0% 


26% 


6% 


Percent on standby 


0% 


55% 


12% 


Percent professional/technical/ 








managerial occupation 


15% 


4% 


13% 


Percent clerical occupation 


16% 


5% 


13% 


Percent manufacturing sector 


22% 


15% 


21% 


Percent services sector 


24% 


27% 


25% 


UI Wages in prior year ($) 


$13,743 


$19,270 


$14,901 


UI Entitlement 








Mean weekly benefit amoimt ($) 


$144 


$183 


$152 


Mean maximum benefits payable ($) 


$3,737 


$5,003 


$4,002 


Site 








Percent in Vancouver 


24% 


39% 


27% 


Percent in Olympia 


12% 


20% 


14% 


Percent in King Coimty 


26% 


20% 


25% 


Percent in Snohomish Coimty 


10% 


11% 


10% 


Percent in Wenatchee 


14% 


3% 


12% 


Percent in Yakima 


14% 


7% 


12% 
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The second step in the intake process involved recruiting targeted claimants for the 
demonstration. Of the 42,350 targeted new claimants who received an invitation to attend an 
Awareness Day meeting, 3,167 (7.5 percent) were interested enough in the possibility of 
participating in the self-employment program to attend. The take-up rate differed somewhat by 
site, from a low of 4.6 percent in Wenatchee to a high of 9.8 percent in Snohomish. Overall, 
the take-up rates were lower in the rural areas of Wenatchee and Yakima. These low rates in 
the Wenatchee and Yakima sites relative to other sites may be indicative of a lower interest in 
self-employment among targeted new UI claimants in rural areas than in urban areas. 
Alternatively, it may reflect the much higher unemployment rates in these rural dreas and 
represent claimants' assessments of the prospects for self-employment in such environments. 

At the end of the Awareness Day meeting, interested claimants were provided with a 
SEED application packet. Of the 3,167 targeted claimants who attended the Awareness Day 
meeting, 1,932 (61 percent) chose to submit a SEED application.'^ Combining the results of 
the first two intake steps - Awareness Day and SEED application - 4.6 percent of all targeted 
claimants who received an invitation letter submitted a SEED application. 

The third step in the SEED intake process was the review of SEED applications. The 
applications were reviewed for timeliness (they were required to be postmarked within seven 
days of Awareness Day) and for being substantively complete. This review process resulted in 
rejecting very few applicants at this stage as the applications were generally quite detailed and 
of high quality. Specifically, of all applications submitted, only 52 (2.7 percent) were rejected 
because they were submitted late and just 20 (1.0 percent) were rejected because they were not 
complete. 

Among those with valid applications, the only remaining reason for exclusion from the 
random assignment pool relates to UI eligibility. In particular, the primary reason for 
application rejection was nonmonetary ineligibility at the time of random assignment; 285 
claimants who submitted applications were determined to be nonmonetarily ineligible for UI 



" Although we have no detailed information on the re^asons for why claimants who attended Awareness Day 
did not submit applications, limited information from the pilot study suggests that the Awareness Day meeting and 
the application served as a useful self-screenmg function. Specifically, the reasons given by a small sample of 
claimants who attended the meeting during the pilot study but who did not submit an application were primarily 
related to lack of adequate capital or concern over whether sclf-cmploymcnt was right given their situation. 
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benefits. Another 55 applicants were rejected because the claim was not monetarily valid and 
a few others were rejected because the UI claim was canceled. 

As a result of these exclusions, a total of 425 of the 1,932 SEED applications submitted 
were excluded from the pool for random assignment, leaving a final pool of 1,507 claimants. 
The random assignment pool corresponds to 47.6 percent of all individuals who attended 
Awareness Day and 3.6 percent of all claimants in the target group. Over the course of the 
demonstration, a total of 755 claimants were randomly assigned to the treatment group and 752 
to the control group. 

A critical evaluation issue concerns the comparability of the individuals in the treatment 
and control groups. We examined the characteristics of these groups in an earlier report (Benus, 
Johnson, and Wood, 1993) and the results indicate that the random assignment process was very 
successful in generating two groups that were quite similar on all of the standard characteristics 
collected at the time the claim was filed. In addition, the groups are extremely similar on prior 
earnings, as well as key items obtained on the SEED application, including prior business 
experience, marital status, family status, reason for job separation, assets, and liabilities. We 
conducted t-tests of differences in means on these characteristics; none of the differences was 
statistically significant at the .05 level. 

Timing of Intake Activities 

The SEED Demonstration was intended to be an early intervention program. It was 
anticipated that by recruiting claimants for SEED as early as possible in their claim and by 
providing services early, the program would provide the maximum possible support to 
individuals during the business startup period. The program was designed to select individuals 
into the treatment group by the fourth or fifth week of the claim and to provide business training 
to treatment group members by the fifth or sixth week. Below we briefly present data that show 
how these timing objectives were achieved in the SEED Demonstration. 

Data on the timing of intake activities through assignment to the first service ~ business 
training modules ~ are summarized overall and by site in Table 3.2. This table shows that the 
intake and recruitment processes occurred as planned. For example, the average length of time 
from the effective date of claim (EDC) until Awareness Day was 18 days, or about 2.5 weeks. 
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About 37 percent of the treatment group members attended an Awareness Day within 12 days 
of their EDC and another 47 percent attended Awareness Day between 13 and 25 days after their 
EDC. Because a few claimants were re-scheduled for a later Awareness Day and some meetings 
were postponed because of holidays, a relatively small percentage (16 percent) did not attend 
a meeting until about 4 weeks after their EDC. Moreover, there were relatively small 
differences in timing across sites. 

The time from Awareness Day to random assignment took another 1 1 days on average. 
Thus, individuals in the treatment group were randomly assigned within 29 days from their EDC 
on average, or within about four weeks in total. There was relatively little variation across sites 
in the time from Awareness Day to random assignment. 

Finally, in the bottom panel of Table 3.2 we show the length of time from random 
assignment to the scheduled daie of the first training module. These data indicate that over 95 
percent of the treatment group members were scheduled to attend their first business training 
module within two weeks of random assignment, with a mean of 10 days. 

Taken together, these data indicate that the average time from EDC to the date for the 
first training module was 39 days or about five and one-half weeks. Thus, it appears tliat the 
timing and frequency of key intake and service activities described in Chapter 2 occurred on 
schedule and that the goal of early intervention was achieved. 

Business Support Services 

As described in Chapter 2, individuals randomly assigned to the treatment group were 
offered a number of business support services as well as financial assistance. The business 
startup services component included intensive classroom training, assistance in preparing a 
business plan, individual counseling, and peer support groups. The financial assistance included 
periodic self-employment allowance payments equal to their weekly benefit amount and a waiver 
of the work search requirement while they were trying to start a business. Moreover, those who 
met all program milestones received a lump-sum payment equal to their remaining UI 
entitlement. 
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The extent to which SEED treatment group members dropped out or completed the 
program and the specific services they received are important to understanding the results of the 
impact analysis. In the remainder of this chapter, we examine these factors and describe their 
SEED experiences. 

Business Training Modules 

Business startup assistance offered to treatment group members began with business 
training modules. Instructions for attending a set of four business training modules at a specific 
location were included in the letter sent to treatment group members informing them of their 
selection into SEED. As described above and in Table 3.2, the first training module was held, 
on average, about 10 days after random assignment. Although attendance at the first module 
was required — in part to ensure that the participation agreement was signed — it was possible 
for subsequent modules to be waived by the business development specialist (BDS) if the 
claimant could demonstrate proficiency in the topics covered in these modules. Treatment group 
members who did not attend the first training module were dropped from the demonstration." 

In Table 3.3, we provide summary information on business training module attendance 
and receipt of other SEED services. Of the 755 claimants in the treatment group, 640 (85 
percent) attended the first training module. This corresponds to a 15 percent dropout rate from 
the treatment group prior to business training." Among treatment group members who 
attended the first business training module very few waivers were granted and nearly all attended 
the remaining three modules. Thus, the attendance rate for all four modules (counting the 
waivers as attenders) was 83 percent of all treatment group members.'^ 



All members of the original treatment and cofltrol groups, including treatment group members who dropped 
out of ths program, were retained in the evaluation to maintain the comparability of the treatment and control 
groups. 

" The attendance rate at the first module varied across sites from 80 percent in Wenatchee, Yakima, and 
Olympia to 88 percent in King County. 

" The overall module attendance rate ranged from 76 percent in Wenatchee to 87 percent in King County. 
The relatively high dropout rate for claimanu in Wenatchee is consistent with the high proportion of claimanu (over 
50 percent) in that site who indicated on their SEED application that they expected to be called back to work by 
their previous employer. 
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Table 3.3 
Washington 
SEED Business Assistance Services Received 



Business Assistance Services 


All Treatment Group 
Memners vIN=755; 


Business Training Modules 




Attenaeo Moauie i 


OJ A) 


Attendea (.or waived; ah Modules 


OJ A) 


Business Counseling Hours 




None 


30% 


. i-.y 


ly A) 


1-1.9 


/I % 


2-2.9 


i 1 of 

13% 


3-3.9 


o of 
6% 


4-4.9 




5 or more 




•Mean Hours of Counseling 


1.5 


Number of Entrepreneur Club Meetings Attended 




None 


64% 


1 


18% 


2 


9% 


3 


5% 


4 or more 


5% 


Mean Number of Meetings Attended 


0.7 



Business Counseling 

In their role as case managers, business development specialists (BDSs) provided 
assistance to treatment group members in the form of counseling on the preparation of a business 
plan and on other issues. The intent was for the EDS to take a proactive role and provide 
individualized counseling and assistance throughout the development of the business plan and 
business startup. To encourage this proactive role, an objective was established for the BDSs 
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to make at least one follow-up contact with all participants during the first few weeks of program 
participation. 

The second panel of Table 3.3 contains the distribution of total business counseling hours 
received by SEED participants. As this table indicates, 70 percent of the treatment group 
received some counseling, with an overall mean of 1.5 hours." The results also indicate that 
very few participants received a substantial amount of counseling, with only 1 8 percent receiving 
3 hours or more.'* As described in Johnson and Leonard (1991), the major focus of the 
individual counseling sessions was on helping treatment group members develop a business plan. 
Overall, nearly one-half of the counseling activities recorded were focused on business plan 
development assistance. 

Other Business Support Services 

In addition to the business training modules and individualized counseling, the SEED 
Demonstration included a peer support group in each site, called the Entrepreneur Club. 
Beginning in the second or third month of the demonstration - after a sufficient number of new 
treatment group members were available - Entrepreneur Club meetings were scheduled on a 
monthly basis. As indicated in the third panel of Table 3.3, the majority of treatment group 
members did not take advantage of this optional peer-support group. Specifically, nearly two- 
thirds (64. 1 percent) of all treatment group members did not attend any Entrepreneur Club 
meetings. Moreover, the mean number of meetings attended was 0.7. Among the 36 percent 
of participants who attended at least one meeting, the mean number of meetings attended was 
about two. 

A final type of business startup service available was referral to other agencies for 
assistance as needed. Vei7 few referrals to other agencies occurred during the SEED 
Demonstration. Specifically, administrative records suggest that there were only 43 referrals 



When calculated over the 630 treatment group members who completed all four training modules, the average 
hours of counseling increases to about 1.8. 

" There were differences in the number of counseling hours by site. For example, about 43 percent of the 
treatment group members in Vancouver received at least 3 hours of counseling, as compared to none in Snohomish 
County, and fewer than 5 percent in Olympia or Wenatchec. Overall, it seems that there was relatively little 
counseling in Olympia, Wenatchec, and Snohomish County, with an average of 0.5-0.9 hours per demonstration 
participant. 
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to Other agencies; 37 of these records were for the Vancouver site, and the remaining six were 
for claimants in King and Snohomish counties. The low frequency of this activity is consistent 
with the views of some BDSs that participants did not need any assistance other than that which 
they were receiving from SEED. 

Financial Assistance 

In addition to receiving their regular weekly UI benefit amount, SEED treatment group 
members received two other forms of financial assistance: a waiver of the work search 
requirement while working full-time to start a business, and a lump-sum payment equal to the 
remaining entitlement at the time all five milestones were met. In this section, we describe 
SEED Demonstration experiences with these financial assistance elements. 

Work Search Waiver 

The work search waiver enabled SEED participants to pursue their business startup plans 
full-time rather than actively searching for employment, as is normally required for UI 
recipients. The duration of the work search waiver was initially set at 10 weeks. Several weeks 
before the waiver was to expire, treatment group members who had not met all milestones were 
sent a letter instructing them to contact their BDS for an End of Waiver Period Review to assess 
their progress in achieving the milestones and to determine whether the work search waiver 
would be extended. There were three possible results of the review: extend the waiver; not 
extend the waiver (and return the claimant to the UI system to search for regular employment); 
or determine that the claimant had met all of the milestones and approve the lump-sum payment. 

The most striking feature of the End of Waiver Period Reviews was the limited extent 
to which such reviews were conducted at all. This was primarily due to the relatively short time 
between program entry and receipt of the lump-sum payment (see next section). Overall, only 
93 claimants (i.e., 12 percent of the treatment group) were recorded as having at least one End 
of Waiver Period Review. Among those for whom a review was conducted, 75 percent occurred 
between 9 and 12 weeks after random assignment as planned, and a large majority (77 percent) 
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resulted in the waiver being extended, with very few claimants (4 percent) being instructed to 
return to regular UI. 

Lump-Sum Payment 

To receive a lump-sum payment, SEED participants had to achieve five milestones: 

• Completion of the training sessions; 

• Development of an acceptable business plan; 

• Establishment of a business bank account; 

• Satisfaction of all licensing requirements; and, 

• Obtaining adequate fmancing. 

In Table 3.4, we provide summary information on the receipt of lump-sum payments in the 
demonstration. During the demonstration, a total of 451 treatment group members (60 percent) 
completed all milestones and received a lump-sum payment." Approximately $1.9 million was 
paid in the form of lump-sum payments, with an average payment of $4,225. 

As shown in Table 3.4, the amount of the lump-sum varied considerably among SEED 
participants. For example, the minimum lump-sum payment was $561 and the maximum was 
$7,380. One quarter of the recipients received a lump-sum payment of less than $3,077, while 
another quarter received more than $5,451. The large differences in the lump-sum payment" 
reflect differences in the maximum benefas payable and in the weekly benefit amount, as well 
as differences in the time required to meet the five program milestones and start a business. ^° 

Table 3.4 also provides information on the length of time after random assignment it took 
for individuals to receive their lump-sum payments. As this table indicates, 44 percent of all 
treatment group members who received their lump-sum payment received it within 6 weeks of 
random assignment (or within approximately 4.5 weeks after the business training modules). 
Another 18 percent took more than 12 weeks after random assignment to complete all of the 
milestones required to receive the lump-sum payment. Overall, the average length of time after 



" The proportion of treatment group menibcrs who received a lump-suni payment ranged from a low of 47 
percent in Wenatchee to a high of 65 percent in Snohomish County. 

* The average lump-sum payment also varied considerably across site and tended to be much higher in the 
urban areas of King and Snohomish County and lower in the rural sites. This difference reflects the higher UI 
entitlement amounts in the urban sites, as well as differences across sites in the time required to meet the five 
milestones. 
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Table 3.4 
Washington 
SEED Lump-Sum Payments 



Lump-sum Payments 


AU Lump-Sum 
Recipients 
(N = 451) 


Amount of Lump-Sum Payment ($) 




Minimum 


$561 


Lowest Quartile 


$3,077 


Median 


$4,360 


Third Quartile 


$5,451 


Maximum 


$7,380 


Average 


$4,225 


Time from Random Assigmnent to Lump-Sum 




Payment (percent) 




Less than 3 WeeLs 


9% 


3 - 6 Weeks 


35% 


6 - 9 Weeks 


22% 


9 - 12 Weeks 


17% 


12 - 15 Weeks 


12% 


More than 15 Weeks 


6% 


Average Number of Weeks to Lump-Sum Payment 


7.8 



random assignment until receipt of the lump-sum payment was 7.8 weeks. Since it took 
approximately 4 weeks on average from the effective date of claim to random assignment, this 
indicates that treatment group members who received a lump-sum payment did so on average 
within about 12 weeks after their effective date of claim. ^' 



2' There were some site differences in the time taken to achieve the lump-sum payment. For example, 
treatment group members who received a lump-sum payment in Vancouver, King County, or Wenatchee averaged 
nearly 9 weeks from random assignment, as compared to roughly 6 weeks in the other three sites. This suggests 
potential differences across sites in the types of businesses established, the needs of claimants, or the ways in which 
BDSs assessed the achievement of milestones. Early in the demonstration, wc identified a large proportion of SEED 
participants in Snohomish County receiving their lump-sum payment extremely early. This occurred because the 
BDS in that site initially treated the milestone review process as pro forma and approved many claimants for ihcir 
lump-sum payment at the end of the business training modules. This situation was identified during site visits and 
corrected at an early follow-up training session. 
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MASSACHUSETTS 
DEMONSTRATION IMPLEMENTATION 



In this chapter we provide an overview of the Massachu:>etts Enterprise Project 
implementation experiences. As we did earlier for the Washington demonstration, we first 
describe the flow of claimants through the intake process — from the identification of targeted 
claimants through random assignment. The effect of the targeting criteria and self-selection is 
then examined by comparing the demographic characteristics of the UI population, the targeted 
population, and the treatment group. 

Following the assessment of sample characteristics, we provide a detailed description of 
the changes in demonstration sites and program operations that took place over time. Following 
this description of implementation changes, we present evidence of the effect of these changes 
on participation in program activities and on the timing of program activities. 

We conclude the chapter by describing the changes in UI regulations that were initiated 
during the demonstration implementation period. These exogenous changes in the demonstration 
environment clearly indicate a strong potential for influencing the impact analysis. For this 
reason we discuss the potential effects of these changes on the analysis results. We also discuss 
whether, in light of these effects, it is appropriate to pool the three cohorts for the impact 
analysis. 

Demonstration Intake 

The Massachusetts Enterprise Project was implemented in three phases. The first phase 
began demonstration intake in May 1990 and continued for approximately five months; the 
second intake phase began in April 1991 , lasting seven months until October 1991; and the third, 
and final, intake phase began in March 1992 and continued for approximately one year, until 
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April 1993. Business support services remained available to demonstration participants for five 
months after the end of each intake period. 

As discussed in Chapter 2, a two-step process was used to identify the target population 
of new m claimants invited to participate in the demonstration. The first step was to apply 
targeting criteria to all new UI claimants and eliminate those who 

• were under age 18; 

• filed interstate claims; and, 

• were eligible for fewer than 26 weeks of Ul benefits. 

These characteristics were identified using information contained in the UI claims record. The 
State computer system identified those new claimants in the categories described above and 
eliminated them from the pool of potential project participants. The claims records for the 
remaining eligible claimants were then transferred from the State UI mainframe to the 
demonstration database, the Participant Tracking System (PTS). (This system is described in 
detail in Chapter 6). 

Since the authorizing legislation required that the program focus on unemployed 
workers likely to exhaust their UI benefits, the second step in identifying the target population 
was to select those individuals likely to exhaust their UI benefits. To do so, we developed an 
algorithm to predict each claimant's likelihood of UI benefit exhaustion.^^ The algorithm 
predicted the probability of UI benefit exhaustion on the basis of information available on the 
UI claims record, including: 

• Total number of dependents; 

• Ratio of benefit amount to average wage; 

• Whether the claimant was permanently separated from the pre-layoff job; 

• Office in which the claim was filed; 

• Education level (four categories - less than ninth grade, some high school, 
college, and postgraduate education); 

• Industry of most recent job (construction, manufacturing, and nondurable 
manufacturing); and, 



^ For a deUiled description of the algorithm, sec Benus, ct al. "MassachusctU Ul Self-Employment 
Demonstration Interim Report to Congress" in Self-Employment Programs for Unemployed Workers, U.S. 
Department of Labor, Employment and Training Administration, 1992. 
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• Occupation (professional or clerical). 
A numerical probability of exhaustion was calculated for each new claimant by applying the 
algorithm to the characteristics of each individual record downloaded from the State mainframe. 
Claimants with a low predicted probability of exhausting UI benefits (i.e., those with a 
calculated probability below .25) were excluded from the target population for the 
demonstration. Those in the target population were then sent an invitation to attend the 
Information Session. 

Exhibit 4.1 shows the distribution of the probability of benefit exhaustion (by cohort) for 
all sample members whose exhaustion probability was .25 or higher. Based on administrative 
data, we estimate that applying the .25 threshold resulted in eliminating 12 percent of all new 
UI claimants in Massachusetts who satisfied the initial targeting criteria. Among those included 
in our sample, the median exhaustion probability was approximately .55 and remained consistent 
across all three cohorts. 

In Table 4.1, we present the affect of applying the above targeting criteria in 1990. 
Specifically, we compare the characteristics of all UI claimants in the demonstration sites with 
those who were invited to attend the Information Sessions (i.e., the target group). As indicated 
by this comparison, new claimants invited to attend the Information Session were similar to the 
population of all UI claimants in the demonstration sites on most characteristics. Both groups, 
for example, were similar in terms of age and race and showed only slight differences in prior 
occupation and industry. In tenns of gender, however, larger differences were found between 
invitees and UI claimants: 60 percent of all UI claimants were male, while only 51 percent of 
those invited (the target group) were male. 

Table 4. 1 also provides infonnation for comparing the characteristics of the target group 
(i.e., those invited to the Information Sessions) with the characteristics of the treatment group. 
This comparison allows us to examine the effects of self-selection into the demonstration. That 
is, since only those UI claimants who were interested in self-employment attended the 
Information Session, and since only the most interested attendees subsequently submitted an 



^ In the two largest sites, Lowell and Wobum, further subsampling was necessary to ensure an even flow of 
participants from all sites. During 1990 and for part of the 1991 cohort, only 75 percent of all likely exhaustees 
were randomly selected to receive an invitation. This subsampling was discontinued in July, 1991. 
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Exhibit 4.1 

Probability of UI Benefit Exhaustion by Cohort 
Massachusetts Demonstration 
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Table 4.1 
Massadiusetts 
Characteristics of UI Claimants, 
Targeted Claimants and the Treatment Group 
Cohort 1 





All UI Qalmants. 


Tai^eted 


Treatment 


Characteristics 


Demonstration Sites 


Claimants 


Groap 




(N== 16,444) 


(N=10,552) 


(N=105) 










Male 


60% 


51% 


65% 


Female 


40% 


49% 


35% 


Age ai Kanuom Assignnieni 








less in oil AO 




16% 


o of 

3% 


25-54 


73% 


73% 


93% 


55 or older 


12% 


11% 


5% 


Mean Age 


NA 


37 


39 


Race/Ethnicity 








Caucasian 


84% 


81% 


87% 


African American 


10% 


12% 


11% 


Hispanic 


5% 


5% 


1% 


Other 


2% 


1% 


. 1% 


Education 








High School Graduate 


NA 


66% 


54% 


Some College 


NA 


16% 


40% 


Mean Education (in years) 


NA 


13 


14 


Former Occunation 








Professional/Technical 


22% 


27% 




Clerical 


17% 


22% 


10% 


Former Industry 








Manufacturing 


26% 


30% 


24% 


Services 


22% 


25% 


37% 


Demonstration Site 








Gloucester 


6.2% 


7.8% 


6.6% 


Greenfield 


4.6% 


6.4% 


9.5% 


Lowell 


24.4% 


22.3% 


22.8% 


New Bedford 


18.9% 


15.4% 


8.6% 


Roxbury 


8.3% 


10.0% 


6.6% 


Springfield 


18.4% 


21.2% 


20.9% 


Wobum 


19.2% 


16.8% 


24.7% 



Source: PTS and DET's Statewide Summary of Claimant and Jobs Data: September, 1990 
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application form, the treatment group is composed of individuals who, through their initiative 
and persistence, selected themselves into the group eligible for random assignment into the 
program. 

As seen in Table 4.1, the target group is nearly evenly divided between males (51 
percent) and females (49 percent); the treatment group, on the other hand, is nearly two-thirds 
male (65 percent). On average, the treatment group is slightly older than the target group (39 
years old versus 37 years old). An examination of the age distribution of the two groups reveals 
that this age difference is largely due to the relative absence of young people from the treatment 
group (only 3 percent of the treatment group was less than 25 years old as compared with 16 
percent for the target group). The treatment group is also more educated than the target group, 
with a higher average number of years of schooling completed (14 years versus 13 years) and 
a substantially higher percentage of individuals who attended at least some college (40 percent 
versus 16 percent). Treatment group members are also more likely than target group members 
to have previously worked in professional/technical occupations (62 percent versus 27 percent) 
and more likely to have previously worked in the services industry (37 percent compared with 
25 percent). Finally, treatment group members are less likely than the target group to be 
employed in clerical occupations and in manufacturing industries. In summary, the self-selection 
of individuals into the demonstration resulted in creating a participant group that may be viewed 
as "more advantaged" than the target population: slightly older, more educated, and more likely 
to be in professional occupations. 

Tables 4.2 and 4.3 present similar data for Ul claimants, target group members, and 
treatment group members in Cohorts 2 and 3. An analysis of these data reinforce the above 
findings. Thus, targeting resulted in a group of eligible claimants who were similar in many 
respects to the general Ul claimant population. Those who self-selected themselves into the 
demonstration, on the other hand, differed from the general Ul population on a number of 
dimensions and yielded a participant group that was more advantaged than the broader target 
group. 
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Table 4.2 
Massachusetts 
Characteristics of UI Claimants, 
Targeted Claimants and the Treatment Group 
Cohort 2 



Characteristics 


All UI Claimants 


Targeted 


Treatment 




jLWuioiiSirauon oii-fss 


^^uunianis 


Group 










Gender 








Male 


CO (It 

58% 


CI Of 

51 % 


70% 


Female 


42% 


49% 


30% 


Age at Random Assignment 








less than 25 


14% 


15% 


2% 


25-54 


75% 


74% 


88% 


55 or older 


12% 


12% 


10% 


Mean Age 


NA 


38 


41 


Race/Ethnicity 








Caucasian 


TO Of 


or\ Of 

80% 


c\ \ of 


African American 


11% 


13% 


8% 


Hispanic 


5% 


6% 


0% 


Other 


.8% 


1% 


1% 


Education 








High School Graduate 


NA 


66% 


58% 


Some College 


NA 


18% 


39% 


Mean Education (in years) 


NA 


13 


14 


Former Occupation 








Professional/Technical 


23% 


27% 


49% 


Clerical 


17% 


18% 


11% 


Former Industry 








Manufacturing 


24% 


24% 


29% 


Services 


19% 


25% 


20% 


Demonstration Site 








Greenfield 


5.0% 


5.7% 


6.3% 


Lowell 


11.3% 


12.3% 


20.8% 


Milford 


24.9% 


21.4% 


20.2% 


New Bef ford 


14.1% 


10.7% 


7.6% 


Roxbur 


10.2% 


11.6% 


5.7% 


Springfield 


17.0% 


20.7% 


17.7% 


Wobum 


17.6% 


17.5% 


21.5% 



Source: PTS ad DET's Statewide Summary of Claimant and Jobs Data: September, 1991 
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Table 4.3 
Massachusetts 
Characteristics of UI Claimants, 
Targeted Claimants and the Treatment Group 
Cohort 3 





All VI Claimants 


Targeted 


Treatment 


Characteristics 


Demonstration Sites 


Claimants 


Group 




(N= 13.126) 


(N=37.7S1) 


(N=351) 


Giender 








Male 


60% 


57% 


72% 


Female 


40% 


43% 


28% 


Age at Random Assignment 








less than 25 


10% 


13% 


2% 


25-54 


77% 


75% 


90% 


55 or older 


13% 


12% 


8% 


Mean Age 


NA 


38 


41 


Race/Ethnicity 








Caucasian 


80% 


80% 


89% 


African American 


12% 


13% 


8% 


riispanic 


J 70 


07T7 




\JukT 






1 % 


Education 








Hign ocnooi urafluate 


MA 
IN A 


07 70 




oome college 








Mean naucation {in years; 




1.3 


14. 

It 


Former Occupation 








Professional /Technical 


28% 


26% 


60% 


Clerical 


19% 


17% 


7% 


Former Industry 








Manufacturing 


27% 


26% 


22% 


Services 


24% 


25% 


30% 


Demonstration Site 








Greenfield 


5.8% 


5.2% 


5.9% 


Lowell 


25.9% 


24.4% 


17.9% 


Milford 


11.7% 


11.0% 


18.2% 


Northampton 


6.7% 


7.5% 


10.8% 


Roxbury 


11.6% 


12.2% 


7.9% 


Springfield 


20.3% 


20.3% 


14.2% 


Wobum 


18.0% 


19.4% 


24.8% 



Source: PTS and DET's Statewide Summary of Claimant and Jobs Data: May, 1993 
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Participant Characteristics 

In the previous section, we analyzed the effect of targeting and self-selection on the 
demonstration sample. We concluded that targeting and self-selection had a consistent effect in 
each of the three cohorts. In this section, we examine the characteristics of the resulting 
samples. 

The demographic and other characteristics of sample members in each of the three 
cohorts are presented in TaWe 4.4." As indicated in this table, Cohorts 1, 2, and 3 are quite 
similar: approximately two-thirds are male; the mean age is approximately 41 years old; the vast 
majority is Caucasian; and the average number of years of education is 14. We conducted a 
t-test of differences in means or proportions for each of the characteristics in the table and found 
none of the cohort differences to be statistically significant at the .05 level. Based on this 
analysis, we conclude that Cohorts 1,2, and 3 are quite similar. 

As discussed in Chapter 3, a critical evaluation issue concerns the comparability of the 
individuals in the treatment and control groups. We compared the characteristics of the 
treatment and control groups (results not presented here) and found that the random assignment 
process was very successful in generating two groups that were remarkably similar on all of the 
standard characteristics collected at the time the claim was filed. In addition, the groups are 
extremely similar on prior earnings and prior business experience. We conducted t-tests of 
differences in means on these characteristics and found that none of the differences was 
statistically significant at the .05 level. Thus, we conclude that differences in outcomes observed 
for treatment and control group members can be confidently attributed to the demonstration. 

Changes in Program Implementation 

As described above, the Massachusetts Enteiprise Project was implemented in three 
distinct phases. During the first enrollment period, which operated from May through 
September 1990, 207 individuals were randomly assigned to treatment or control status. This 



For this analysis we combine treatment and control group members. 
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Table 4.4 
Massachusetts 
Characteristics of Experimental Sample 
(Treatments and Controls) 
Cohorts 1, 2, and 3 



Charactttpistics* 

^^MmmMJL VVVV^afK Vh7v>^>47 


Cohort i n 


Cohort 2 
(N«314) 


Cohorts 
(N»701) 


Gender 








Male 


" 67% 


67% 


70% 


Female 


33% 


33% 


30% 


Age at Random Assignment 








Mean Age (in years) 


39.8 


41.1 


40.7 


Race/Ethnicity 








Caucasian 


89% 


90% 


89% 


African American 


9% 


8% 


9% 


Hispanic 


1% 


1% 


2% 


Other 


1% 


1% 


1% 


Education 








Percent College Graduate 


43% 


45% 


45% 


Mean Education (in years) 


14.3 


14.6 


14.5 


UI Entitlement ($) 








Mean Weekly Benefit Amount 


$244 


$250 


$268 


Mean Maximum Benefit Payable 


$7,249 


$7,448 


$7,990 



Source: PTS 



All values shown in the table are based on non-missing values. Therefore, the sample size for different variables may 
vary slightly. 

enrollment period is referred to as Cohort 1 (or the 1990 sample). The Cohort 2 (or the 1991 
sample) enrollment period lasted from April through October of 1991, during which time 314 
claimants were enrolled in the experimental sample. Cohort 3 (or the 1992-93 sample) 
enrollment lasted from March 1992 through April 1993, assigning a total of 701 individuals to 
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treatment and control status. Thus the Massachusetts demonstration includes 1,222 UI 
claimants. 

While the overall project design did not change between program years, some program 
features did change during the three-year implementation period. Since these changes may 
influence and/or help explain the analytic results presented in subsequent chapters, we highlight 
these changes below. 

Changes in Demonstration Sites 

Perhaps the most important change in the Enterprise Project implementation was the 
change in demonstration sites. In 1991, for example, Milford was added and Gloucester was 
dropped from the demonstration. The primary reason for this change was to increase the flow 
of claimants into the demonstration (Milford is substantially larger than Gloucester). The 
remaining sites (Greenfield, Lowell, New Bedford, Roxbury, Springfield, and Wobum) operated 
the demonstration in both 1990 and 1991. 

For the final phase of the demonstration (1992-93), an additional change in sites took 
place: New Bedford was dropped and Northampton was added. This change was necessitated 
by the fact that New Bedford was selected as a site for the Massachusetts Industrial Services 
Program (ISP), another entrepreneurial training program. The selection of New Bedford as an 
ISP site increased the likelihood that Enterprise Project control group members would have 
received entrepreneurial training services through the ISP program, thus, potentially confounding 
the demonstration results. Furthermore, the Massachusetts Department of Employment and 
Training (DET) commissioner detentiined that it would be inappropriate to operate two 
entrepreneurial training programs in one city while many other cities had no entrepreneurial 
program at all. As a result, DET recommended that we drop New Bedford as a site and add 
Northampton." Thus, only five sites (Greenfield, Lowell, Roxbury, Springfield, and Wobum) 
operated the program in all three phases. Milford and New Bedford operated the program for 
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This change had another advantage. Northampton could combine with Greenfield for the biweekly 
Enterprise Workshops, reducing the travel time for Greenfield participants (who previously had to travel to 
Springfield for many sessions). 
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two of the three phases and Gloucester and Northampton for one year. These site changes and 
the location of the demonstration sites are depicted in Exhibit 4.2. 

Changes in Program Operations 

Other significant changes occurred in the delivery of business assistance services. 
Initially, business support services were provided by the Massachusetts Small Business 
Development Centers (MSBDC) and the Franklin County Community Development Corporation. 
Based on feedback from demonstration participants and DET's evaluation of program services 
during the first year, DET determined that a change in service providers was needed to improve 
the responsiveness of business support services to the needs of the clients. Prior to the second 
implementation phase, therefore, DET contracted with four business development organizations 
to provide business support services for Cohort 2 members. Three of the four were retained for 
Cohort 3 and a new provider was added to better serve the Roxbury site. 

While efforts were made to maintain consistency in training services during all three 
phases, some changes were necessary to meet client needs and to improve service delivery. For 
example, the length of the Enterprise Seminar (the first self-employment training session) was 
reduced from eight hours in 1990 to 4.5 hours in 1991. This change was precipitated by 
feedback from participants and business development specialists who indicated that an eight-hour 
session resulted in fatigue for participants and made it difficult for them to absorb all of the 
information presented. To accommodate the shorter timeframe for the Enterprise Seminar, the 
curriculum was altered to eliminate the detailed discussion of financial and personnel issues that 
had been included in the Cohort 1 Enterprise Seminars. ^"^ The eliminated topics were added 
to the bi-weekly session curricula. 

For the 1992-93 phase, the Enterprise Seminar was lengthened slightly to 5.5 hours. 
This change was based on recommendations from the business development specialists who 
found that 4.5 hours was not quite sufficient time for presenting the material. While the 



^ Additionally, the 1990 Enterprise Seminar included an introductory discussion of the Massachusetts Small 
Business Development Centers and their role in the Enterprise Project. This material was not relevant for the 1991 
sessions and was therefore eliminated. 

Go 
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Exhibit 4.2 

Massachusetts Enterprise Project Demonstration Sites 
During the Three Implementation Phases 




Site 


1990 


1991 


1992 


Gloucester 


* 






Greenfield 


* 


* 


* 


Lowell 


* 


* 


* 


Milford 




* 


* 


New Bedford 


* 


* 




Northampton 






* 


Roxbury 


* 


* 




Springfield 


* 


* 


* 


Woburn 


* 


* 


* 



Source: Massachusetts DET 
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curriculum was not altered, slightly more time was allocated for each topic and a lunch break 
was added. 

The curricula and schedules for the bi-weekly sessions" were significantly altered 
following the first year of implementation in order to increase consistency across the 
demonstration sites. For Cohorts 2 and 3, standard curricula were developed for each of the six 
workshops: (1) Cash Flow and Financial Statements; (2) Personal Effectiveness and Selling; (3) 
Financing; (4) Marketing; (5) Legal Issues/Insurance; and, (6) Bookkeeping/Taxes. 

For Cohort 2, the schedule was organized so that participants could attend all six 
workshops over a 12-week period. Beginning in Cohort 3, the schedule for the workshop 
sessions was shortened so that all set sessions could be completed within nine weeks (rather than 
twelve). This change was made to accelerate participants' progress through the demonstration 
activities and to assist participants in starting their businesses more quickly. In addition to 
modifying the schedule for the workshops, DET developed the schedule so that the three 
workshops deemed most critical to early business development ~ Cash Flow and Financial 
Statements, Personal Effectiveness and Selling, and Marketing - were scheduled to occur first. 
DET also established a staggered schedule of workshops throughout the State so that participants 
could quickly make up missed workshops in other locations. Furthermore, each bi-weekly 
workshop was mcreased from 2 to 2.5 hours to allow more time to cover each topic. 

Cohort 3 participants were required to present their business plans to their fellow 
participants during the workshop sessions. This change was made at the suggestion of the 
business development counselors, who felt that requiring an oral presentation would assist 
participants in developing their marketing skills as well as self-assess their progress. 

Another change, instituted for Cohort 3, was the addition of a mid-program review. In 
this mid-program review, the business development counselor and the participant met to evaluate 
the participant's progress toward starting the business. As part of this review, the counselor, 
together with the participant, decided whether or not it was appropriate for the participant to 
continue in the demonstration. It is important to note that maintaining satisfactory progress was 
always a requirement of the demonstration. The addition of the mid-program review can thus 



The name of these sessions was changed in 1991 to Enterprise Workshops. 
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be viewed as a mechanism for ensuring that demonstration participants make satisfactory 
progress towards starting their businesses, although satisfactory progress was always a 
requirement of the Massachusetts demonstration. It should also be noted that the mid-program 
review was similar to the End of Waiver Period/Review in the Washington demonstration. 

Participation in Enterprise Project AcTivmES 

Information on the participation rates in various Enterprise Project activities across the 
three phases is shown in Table 4.5. As indicated in the table, the percentage of individuals who 
attended an Information Session remained fairly constant over time: 3.5 percent in Cohort 1, 
4. 1 percent in Cohort 2, and 4.3 percent in Cohort 3. The applic?.:ion rate (i.e. , the percentage 
of Information Session attendees who submitted an application) dropped markedly over time, 
from a high of 69 percent in Cohort 1 to 53 percent in Cohort 3. The combination of increasing 
attendance rates and decreasing application rates yields a relatively constant overall takeup rate 
(i.e., the rate of participation among those eligible to participate). In Cohort 1, 2.4 percent of 
all invited claimants attended the Information Session and submitted an application. This rate 
remained at 2.4 percent in Cohort 2 and was 2.3 percent in Cohort 3. 

The bottom panel of Table 4.5 provides the rate of participation in business assistance 
services among those eligible to participate. These results indicate substantial similarity between 
Cohorts 1 and 2, with more than 92 percent of all treatment group members attending the 
Enterprise Seminar and approximately 90 percent attending at least one counseling session. 
Cohort 3 results indicate slightly higher attendance rates, with 94 percent attending the 
Enterprise Seminar ai;d 94 percent attending at least one individual counseling session. 

Table 4.5 also indicates a steady increase in the proportion of treatment group members 
attending at least one bi-weekly session: 77 percent in Cohort 1, 90 percent in Cohort 2, and 92 
percent in Cohort 3. This increase in attendance over time may reflect the more formal 
organization of the sessions, the standardization of curricula, and the tightening of the schedule 
for completing all workshops. 
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Table 4.5 
Massachusetts 
Participation In Various Enterprise Project Activities 





Cohort 1 


Cohort! 


Cohort 3 j 


Ac^vity 


Number 


Fofcettt 


Number | 


Vtxteoi 


Number 


Permit 1 


Invited to Information 
Session 


10,552 




15,618 




37,751 




Attended Information 
Session* 


372 


3.5% 


641 


4.1% 


1,645 


4.3% 


Submitted an 
application* 


257 


69% 
(2.4% of 
all invited) 


381 


59% 

A Of ^£ 

(2.4% Of 
all invited) 


877 


53% (2.3% 

r\f oil 

oi ail 
invited) 


KanaonfUy Assignea 
Treatment Group 
Control Group 


207 
105 
102 


81% 
(1.9% of 
all invited) 


314 
158 
156 


82% 
(2.0% of 
all invited) 


701 
351 
350 


80% (1.8% 
of all 
invited) 


Business Assistance 
Services 

Received by Treatment 
Group Members 














Enterprise Seminar 


97 


92% 


146 


92% 


330 


94% 


At least one 
counseling session 


94 


90% 


145 


92% 


330 


94% 


At least one bi- 
weekly workshop 


81 


77% 


142 


90% 


322 


92% 



Source: PTS 



* The pcrcenUges shown are percenUges of the activity above. 
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Tevdng of Enterprise Project AcnvniES 

The interval between key Enterprise Project activities in Cohorts 1, 2, and 3 is presented 

in Table 4.6. For most activities, the elapsed time between consecutive activities was somewhat 

shorter for Cohorts 2 and 3 than for Cohort 1 . For example, the average elapsed time from 

application receipt to random assignment decreased from 10 days in Cohort 1 to 8 days in 

Cohorts 2 and 3. The total number of days between the benefit-year-begin date and attendance 

at the Enteiprise Seminar was 64 days in Cohort 1 and 58 days in Cohorts 2 and 3. 

Table 4.6 
Massachusetts 

Average Niunber of Days Between Key Program Activities 
Cohorts 1, 2, and 3 





Cohort! 


Cohort 2 


Cohort 3 


Activity 


Mean Days 


Mean Days 


Mean Days 


Benefit Year Begin Date to Invitation Date 


22 


23 


23 


Invitation Date to Date of Information Session 


12 


10 


10 


Information Session Attendance to Application Receipt 


6 


6 


5 


Application Receipt to Random Assignment 


10 


8 


8 


Random Assignment to Enterprise Seminar 


14 


11 


12 


Benefit Year Begin Date to Enterprise Seminar 


64 


58 


58 



Changes in UI Regulations 

As discussed in Chapter 1, Section 9152 o^ .ne Omnibus Budget Reconciliation Act 
(OBRA) of 1987 authorized the Massachusetts Demonstration. This legislation required the State 
to reimburse the Unemployment Trust Fund any excess cost resulting from the self-employment 
demonstration. Excess costs arise when demonstration treatment group members, on average, 
collect more self-employment allowances than the amount of UI benefits they would have 
collected in the absence of the demonstration (as measured by the experience of control group 
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members). Thus, excess costs are measured as the difference in benefits paid to treatment and 
control group members. 

As part of the demonstration design phase, the Massachusetts DET, together with DOL 
and the research contractor, developed a variety of demonstration features designed to reduce 
the potential for excess costs while maintaining all the legislative requirements." One 
important design feature was the statistical algorithm to identify likely UI exhaustees. This 
algorithm was used throughout the demonstration to offer Enterprise Project services oniy to UI 
claimants who exceeded a threshold exhaustion probability (as required by the authorizing 
legislation) . A second important design feature was the establishment of a 24- week work search 
waiver period for Enterprise Project participants. As a result of this demonstration design 
feature, treatment group members could collect self-employment allowances only through the 
24th consecutive week of their UI claim. At that point, treatment group members had to choose 
between continuing with their self-employment activities full-time or returning to UI for their 
remaining six weeks of benefits and meeting the regular work search requirements. 

Thus, the design of the Massachusetts Enterprise Project required treatment group 
members to forego up to 6 weeks of UI benefits if they chose to continue their self-employment 
activities full-time beyond the 24th week of their UI claim. With an average weekly benefit rate 
of $244 (in 1990), treatment group members thus faced a tradeoff between continued pursuit of 
self-employment and UI benefits valued at $1,464. Control group members did not have to 
make such a choice since they could collect up to the maximum duration of UI benefits ~ 30 
weeks in Massachusetts. 

Starting in November 1991 and continuing through October 1993, a variety of changes 
dramatically affected the tradeoff for treatment group members. For example, effective in 
November 1991, the Emergency Unemployment Compensation (EUC) Act of 1991 provided for 
up to 20 weeks of additional EUC benefits beyond the 30 weeks of UI benefits available to 
Massachusetts claimants. Subsequent extensions of EUC (there were four additional EUC 
periods) provided for up to 33 weeks of EUC benefits. Thus, during part of the demonstration 



^ Similar design features were not incorporated into the Washington demonstration since the Washington 
demonstration was funded through DOL research funds and, thus, not required to reimburse excess costs. 
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period, treatment group members faced a tradeoff of substantially more than 6 weeks of UI 
benefits. 

While it is impossible to quantify the exact tradeoff faced by treatment group members 
throughout the tliree-year implementation period, we can calculate the initial impact on the 
tradeoff resulting from the passage of the EUC Act. Treatment group members who were 
eligible for EUC in November 1991, faced a tradeoff of 26 weeks (6 weeks of UI benefits plus 
20 weeks of EUC benefits). With an average weekly benefit rate of $250 (in 1991), the value 
of the tradeoff was thus raised to $6,500 by the passage of the EUC Act of 1991. 

The effect of this dramatic increase in the tradeoff facing treatment group members is 
impossible to quantify. It is clear, however, that when facing a substantially higher tradeoff, 
treatment group members are less likely to continue their self-employment activities beyond 
week 24. Thus, we expect to observe fewer treatment group members in Cohort 3 continuing 
with their self-employment effort beyond week 24 as compared with treatment group members 
in Cohorts 1 and 2. Thus, holding other factors constant, smaller program impacts in Cohort 
3 are expected as compared with Cohorts 1 and 2. 

The Massachusetts impact results are also likely to be affected by another change in UI 
regulations. In 1992, the Massachusetts legislature passed a law limiting UI benefits to 26 
weeks (from 30 weeks) during periods when EUC or any other federal extended unemployment 
benefits program was in effect. The impact of this change in the law on the behavior of 
program participants is unknown. The change, however, is expected to affect our analysis of 
UI benefits collected. That is, since the data discussed here include only UI benefits (and 
exclude other benefits such as EUC), the impact of this change is likely to distort our analysis 
of UI benefits receipt. We explore this issue with the aid of a graph below. 

Exhibit 4.3 depicts the number of weeks of UI benefits collected by treatment and control 
group members in each cohort. As indicated in the top panel of the exhibit, approximately an 
equal number of treatment group members in Cohort 1 collected either 24 or 30 weeks of UI 
benefits, while the majority of control group members collected 30 weeks of benefits. This 
pattern is repeated for members of Cohort 2, where approximately an equal number of treatment 
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Exhibit 4.3 

Weeks of UI Benefits by Treatment and Control 

Cohorts 1, 2 and 3 
Massachusetts Demonstration 



Cohort 1 
Treatment Group 




0 5 10 15 20 25 30 35 40 45 50 

Weeks 
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100 




0 5 10 15 20 25 30 35 40 45 50 
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20 



Cohort 1 
Control Group 



10 
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Exhibit 4.4 

Weeks of UI Benefits by Treatment and Control 
Cohorts 1&2 and Cohort 3 
Massachusetts Demonstration 



Cohorts 1 & 2 
Treatment Group 
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group members also collected either 24 or 30 weeks of benefits and the majority of the control 
group collected 30 weeks of UI benefits. 

The pattern, however, is substantially different for Cohort 3 members. As shown in the 
bottom panel of Exhibit 4.3, the distribution of weeks of UI benefits has three peaks for the 
treatment group, and two peaks for the control group. That is, in Cohort 3, the majority of 
treatment group members collected either 24, 26, or 30 weeks of benefits. That is, 128 
collected 24 v/eeks of UI benefits, 94 collected 26 weeks, and 39 collected 30 weeks. Within 
the control group, the majority collected either 26 or 30 weeks of benefits (134 and 56, 
respectively). These results for Cohort 3 suggest that the availability of Federal supplemental 
benefits, together with the change in the UI law, appears to have altered the behavior of Cohort 
3 relative to the earlier cohorts. 

To investigate the results of Cohort 3 further, we split the sample into two groups: those 
who filed a UI claim before June 1, 1992 (i.e., the effective date of the State UI law change) 
and those who filed after that date. In Exhibit 4.4, we present the distribution of number of UI 
weeks collected b} the combined Cohort 1 and Cohort 2 sample (top row). In the second row 
of the exhibit, we present the results for Cohort 3 members who filed before June 1, 1992; the 
third row shows results for those who filed after that date. An examination of this exhibit 
reveals that those who filed before June 1 have a pattern similar to the combined Cohorts 1 and 
2 sample; that is, approximately an equal number of treatment group members collected either 
24 or 30 weeks of UI benefits and a majority of the control group collected 30 weeks of 
benefits. 

Those in Cohort 3 who filed after June 1 (bottom row), showed an entirely different 
pattern. Within the treatment group, approximately an equal number collected either 24 or 26 
weeks of UI benefits. Within the Control group, a majority collected 26 weeks of UI benefits. 
This decomposition of Cohort 3 suggests that it is composed of two distinct subgroups -- those 
who filed an initial UI claim before the effective date of the law change and those who filed after 
that date. Cohort 3 thus operated under two very different environments. 

Based on the above analysis of UI benefits receipt by cohort, can we combine all three 
cohorts for the impact analysis? Alternatively, should we combine part of Cohort 3 (i.e., those 
who filed before the change in the law) with Cohorts 1 and 2? We take the conservative 
approach of estimating impacts separately for Cohort 3, while combining Cohorts 1 and 2. 
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COMPARISON OF 
DEMONSTRATION IMPLEMENTATION 



In the previous two chapters, we presented the implementation experiences of the 
Washington SEED Demonstration and the Massachusetts Enterprise Project separately. In this 
chapter, we compare the implementation experiences of the two demonstrations, in an attempt 
to gain insights that will help explain differences in program impacts. 

In this chapter, we compare the experience of the entire SEED Demonstration with the 
entire Enterprise I'roject Demonstration. For this analysis, therefore, we combine all three 
Massachusetts cohorts and compare their combined implementation results with the results of 
the Washington sample. In the sections below, we first compare the timing of the two 
demonstration intake periods. Next, we compare the participation rates in various demonstration 
activities (e.g., attendance at orientation, application, random assignment, receipt of business 
services). Then, we compare experiences of demonstration participants, their assessment of 
demonstration services, and throughout this chapter, we place special emphasis on identifying 
implementation differences that may lead to program impact differences. 

Timing of tboe Intake Periods 

Perhaps the most important implementation difference between the two demonstrations 
is the difference in intake periods. In Washington, there was a single intake period, from 
September 1989 ti September 1990. In contrast, the Massachusetts demonstration had three 
distinct intake periods. The first intake period started in May 1990 and the last concluded in 
April 1993. Both the Washington and Massachusetts intake periods are depicted in Exhibit 5.1. 

During the single Washington intake period, a total of 1,507 Washington SEED 
applicants were randomly assigned to either the treatment group (755) or the control group 
(752). During the three distinct Massachusetts intake periods, a total of 1,222 Massachusetts 
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Exhibit 5.1 

Demonstration Enrollment Over Time 
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Enteiprise Project applicants were similarly assigned to the either the treatment (614) or the 
control (608) group. Thus, in terms of sample size, the Washington demonstration was larger 
tlian the Massachusetts demonstration (1,507 vereus 1,222). 

As is clear from Exhibit 5.1, the three Massachusetts intake periods lasted substantially 
longer than the Washington iotake period. From the beginning of the first intake period (in May 
1990) to the end of the last intake period (in April 1993) nearly three years had passed. Even 
if we exclude the gaps between the intake periods, the actual intake periods in Massachusetts 
lasted a total of approximately two years as compared with one year for the Washington 
demonstration. 

Table 5.1 

Washington/Massachusetts Comparison 
Participant Flow 





Massa^^husetts 




Washii^ton 


Number 


Percnit 


Number 


Poxent 


Invited to Information 


63,921 




42,350 




Session 








Attended Information 


2,658 


4.2% 


3,167 


7.5% 


Session 








Submitted an application 


1,515 


57.0% 


1,932 


61.0% 






(2.4% of all 




(4.5% of all 






invited) 




invited) 


Randomly Assigned 


1,222 


80.7% 


1,507 


78.0% 


Treatment Group 


614 


(1.9% of all 


755 


" (3.6% of all 


Control Group 


608 


invited) 


752 


invited) 



Source: Participant Tracking System 



Attendance at Orientation 

In Table 5.1, we present a comparison of the participant flow through various stages of 
the demonstrations. This flov» is also depicted in Exhibit 5.2. An examination of Exhibit 5.2 
indicates substantial similarity m the two flow patterns. That is, only a small portion of the 
target groups in both demonstrations chose to attend an information session. This pattern 
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Participation in Various Project Activities 
Massachusetts and Washington 



Massachusetts ■ Washington 
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reinforces the observation that self-employment is not for everyone. In fact, most Ul claimants 
who are offered an opportunity to participate in a self-employment program, choose not to attend 
a meeting where they can get additional information about the self-employment program. 
Among those who expressed an interest (or, at l&'ist, a curiosity) by attending the orientation 
mesting, approximately three-fifths submitted an application and approximately four-fifths were 
accepted and randomly assigned in both demonstrations. 

As indicated in Table 5.1, in Washington, a total of 42,350 UI claimants were invited 
to attend a SEED orientation session; in Massachusetts, a total of 63,921 UI claimants were 
invited to attend an Enterprise Project orientation session (combining the three years of the 
Massachusetts demonstration). The response rate to the invitations was substantially higher in 
Washington than in Massachusetts. In Washington, 7.5 percent of those invited to attend an 
orientation session were interested enough in the possibility of pursuing self-employment that 
they chose to attend the session. In Massachusetts, 4.2 percent of those invited to an orientation 
session attended. The attendance rate in Washington is thus nearly twice the attendance rate in 
Massachusetts. 

There are several possible explanations for the difference in attendance rates. First, the 
contents of the two invitation letters were different and may have influenced the attendance rate. 
Second, claimants may have been invited to the orientation at different stages of their 
unemployment spell. Third, the target populations in the two demonstrations may have differed 
on some characteristic that influences interest in self-employment. Fourth, knowledge of the 
lump-sum payment (in Washington) may have encouraged more people to participate. Finally, 
there may have been other factors, such as local economic conditions that affected the attendance 
rates. W^e examine each of these factors below. 

The content and wording of the invitation letters may have affected the attendance rates. 
In Exhibits 5.3 and 5.4 we present the invitation letters used in the two demonstrations. In the 
Washington letter (Exhibit 5.3), the first sentence asks simply "Are you interested in self- 
employment?" In the Massachusetts letter (Exhibit 5.4), the first sentence asks a somewhat 
more narrow question "Do you have an idea for your own business?" Thus, in Washington, 
individuals with a general interest in self-employment but with no specific business idea were 
invited to attend. In Massachusetts, on the other hand, the wording of the letter targeted the 
invitation to only those individuals with a business idea. This may partly explain why relatively 
fewer individuals in Massachusetts attended the orientation session. 
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Exhibit 5.3 
Washington 
Invitation Letter 



Name SSA Letter 

Address 1 JSC 

Address 2 BYE 

City, State Current Date 
Zip 

Are you interested in self-employment? If so, you are invited to attend a one-hour self-employment 
Awareness Day meeting: 

Date 

Location . 



(Please arrive early, latecomers will not be allowed) 

The State of Washington is conducting a special project for people interested in self-employment. The 
regular unemployment insurance system does not allow claimants to receive benefits and pursue full-time 
self-employment. But the purpose of the Self-Employment & Enterprise Development (SEED) Project 
is to find ways to make the Unemployment Insurance system more effective for people who would like 
to start their own business. 

Participants in the SEED project will receive unemployment benefits and training while preparing to start 
their own business. The project is designed for individuals who already have a business idea and are 
prepared for a quick start. You must also be fully eligible to receive unemployment benefits to 
participate. 

This is a one-time offer,. YOU MAY NOT RESCHEDULE. To learn more about this project and help 
you decide if you wish to apply, you must personally attend Awareness Day on the date that appears 
above. Attendance at this meeting will not affect your eligibility for unemployment benefits. 

If you have questions about the SEED project, please do not contact your Job Service Center. You may 
call the SEED Project Unit at 1-800-782-9099 or 1-206-586-8849. 
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Exhibit 5.4 
Massachusetts 
InvitatioL. Letter 



(Name) 

Pate) 

(Address) 

Dear (Name): 

Do you have an idea for your own business? Would you like to try to turn your idea into a full-time job? 

The Enterprise Project is a pilot program that the Commonwealth of Massachusetts is offering in seven 
selected communities for people who are receiving unemployment insurance and want to start their own 
businesses. To find out about the program and to learn a little about self-employmen<. you r.'iOuld attend 
one of the following Information Session: 



(Date) 
(Time) 



(Site) 
(Address) 



(Date) 
(Time) 



OR 



(Site) 
(Address) 



OR 

■ (Date) (Site) 

(Time) (Address) 

At this meeting, a representative from the Department of Employment and Training will explain how the 
Enterprise Projects ties in with your unemployment comp'>nsation. A business counselor will discuss the 
risks and rewards of self-employment and help you begin to evaluate your business idea. 

To qualify for the Enterprise Project, you must have attended an information session and have a clear 
business idea which will be reviewed for final eligibility. The Project is only open to people who will 
not be recalled to their previous job. The Project is very small. Applicants will be selected by lottery. 
I will be happy to answer any questions you may have about the program at the information session. 



Sincerely, 



(Enterprise Rep. Nime) 
Enterprise Representative 
(Site) Opportunity' Job Center 
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Moreover, the Massachusetts letter mentions that the Enterprise Project is very small and 
that applicants will be selected by lottery. The Washington letter, on the other hand, makes no 
mention of random selection or limited enrollment. The Washington letter may thus be viewed 
as less restrictive and somewhat more encouraging to invitees than the Massachusetts letter, 
timing of the invitation letter and the scheduling of the orientation sessions may have also 
influenced the attendance rates in the demonstrations. As described earlier, the time interval 
between the benefit year begin date and the orientation attendance date was shorter in 
Washington than in Massachusetts (by approximately 2 weeks)." If some Massachusetts 
claimants found jobs during the extra interval, they would not have attended the orientation 
session. Given the high average duration of unemployment in Massachusetts, however, it is 
unlikely that the extra two weeks had a significant impact on the attendance rate. 

An examination of the demographic characteristics of the target populations in the two 
demonstrations (shown for Washington in Table 3.1 and for Massachusetts in Tables 4.1, 4.2, 
and 4.3) shows that the Massachusetts target group (those sent invitations) are more educated, 
slightly older, and more likely to have previously worked in professional/ technical occupation 
and in the manufacturing industry as compared to the Washington target group. 

There is some anecdotal evidence that individuals in Washington filed claims in 
demonstration sites in order tc be eligible for the demonstration and, more specifically, the 
lump-sum payment. During the pilot phase, for example, when Vancouver was the only 
demonstration site, some individuals actually traveled from Seattle to Vfdicouver (180 miles) to 
file a claim. To the extent that this occurred, it would contribute to the higher response rate in 
Washington. 

Finally, economic conditions in Washington were relatively -)bust during the SEED 
enroUment period. In Massachusetts, on the other hand, economic conditions were very poor 
throughout the demonstration. Prior evidence suggests that the local unemployment rate is 
positively related to interest in self-employment (i.e., higher unemployment rates are related to 



^ The reason for this difference is largely a result of the fact that, during the demonstration, Washington was 
a wage reporting state while Massachusetts was not. This resulted in a longer period of time needed to determine 
monetary eligibility for UI in Massachusetts. Following the conc> .^ion of the demonstration, Massachusetts has 
also become a wage reporting state. 
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increased interest in self-employment). Thus, economic conditions do not help to explain the 
higher attendance rates in Washington. 

Application 

After attending the initial orientation session, the next step in the intake process in both 
demonstrations was filing an application. At the end of the initial session, those interested in 
applying to the demonstration took an application packet and were required to submit the 
completed package within seven days. As indicated in Table 5.1, the application rate among 
information session attenders in Washington was only slightly higher than in Massachusetts (61 
percent versus 57 percent). Thus, it appears that although fewer people in Massachusetts had 
interest in the demonstration at the beginning of the process, those who completed the first step 
were ahnost as likely as their counterparts in Washington to proceed to the next step - 
application. 

Random Assignment 

In both projects, applications were reviewed by demonstration staff to ensure timely 
submission, completeness and that the business ideas satisfied the guidelines of the 
demonstration. Those applications that met the criteria were then randomly assigned. In 
Massachusetts, 80.7 percent of all applications submitted were randomly assigned, whereas in 
Washington 78.0 percent of submitted applications were randomly assigned. As indicated in 
Table 5.1, in Washington the takeup rate (i.e., the rate of participation among those eligible to 
participate) was 3.6 percent as compared with only 1.9 percent in Massachusetts. This 
substantial takeup rate difference in the two demonstrations is due largely to the difference in 
the Information Session attendance rates. 

Timing of Intake Acttvities 

Both demonstrations were intended to be early intervention programs. It was anticipated 
that by recniiting claimants early in their UI claim, and by providing self-employment training 
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and counseling services as early as possible, the programs would provide the maximum possible 
support to individuals during the business startup period. 

In the Washington demonstration, early delivery of program services also had an 
important monetary incentive. That is, demonstration participants were required to complete 
five project milestones in order to receive a lump-sum payment of their remaining UI benefits. 
To maximize this amount, it was important to complete the training and develop a business plan 
as soon as possible. 

Table 5.2 shows the average number of days between key intake and service activities 
in the Washington and Massachusetts demonstrations. The total number of days between benefit 
year begin date/effective date of claim and attendance at the orientation session was 33 days in 
Massachusetts and only 18 days in Washington. This difference is largely explained by the fact 
that Washington, as a wage reporting state, could complete monetary eligibility determination 
more quickly than Massachusetts, a wage request state. This procedural difference resulted in 
the observed 15-day difference between the two demonstrations. 

Table 5.2 

Washington/Massachusetts Comparison 
Average Number of Days Between Key Program Activities 
(All Treatment Group Members) 



Activity 


Washington 
(N»«755) 


Massadiust;^ 
(N«614) 


Benefit Year Begin Date/Effective Date of Claim 
to Information Session/ Awareness Day 


17.7 


32.8 


Orientation Session to Random Assignment 


11.1 


13.4 


Random Assignment to Enterprise Seminar/First 
Training Module 


10.2 


13.1 


Benefit Year Begin Date/Effective Date of Ciaim 
to Enterprise Seminar/First Training Module 


39.0 


59.0 



Source: PTS 
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The elapsed time between the orientation session and random assigi iment was also slightly 
longer in Massachusetts as compared with Washington (13 days versus 1 1 days). Similarly, the 
elapsed time between random assignment and attendance at the first training sessions was longer 
in Massachusetts than in Washington (13 days versus 10 days). 

The cumulative effect of each of tlie above delays in Massachusetts relative to 
Washington is measured by comparing the total elapsed time between the benefit begin date and 
the date of the first busmess support activity (Enterprise Seminar/First Training Module). The 
elapsed time on this measure is 20 days longer in Massachusetts than in Washington (59 days 
compared to 39 days). Thus, on every measure examined, services were provided more rapidly 
in the Washington demonstration than in the Massachusetts demonstration. 

Table 5.3 

Washington/Massachusetts Comparison 
Business Assistance Services Received by the Treatment Group 
(Percentages based on PTS data) 





Washington 


Massachusetts 


Business Assistance Services 








(N-755) 


(N=614) 


Business Training Modules 






Attended Enterprise Seminar/Module 1 


84.8% 


93.3% 


Attended At Least Six Bi-Weekly 






Workshops/ Attended (or Waived) All 






Modules 


83.4% 


49.7% 


Business Counseling Hours 






No counseling 


29.9% 


7.3% 


Mean Hours of Counseling 


1.5 


7.5 


Number of Entrepreneur Club Meetings Attended 






None 


64.1% 


NA 


Mean Number of Meetings Attended 


0.7 


1 NA 



Source: PTS 
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Program Experiences 

In Table 5.3 we provide information on the receipt of business assistance services by 
demonstration participants. As shown in the table, a higher percentage of Massachusetts 
treatment group members (93 percent compared with 85 percent) attended the first activity after 
random assignment. However, a much smaller proportion of Massachusetts participants attended 
all the required workshop sessions. In Massachus^^tts, only approximately half of the treatment 
group attended all six Enterprise workshops; in Washington, 83 percent of the treatment group 
attended the four required training sessions. This difference is partly explained by the difference 
in timing of the workshops in the two demonstrations. That is, in Washington, the four training 
modules were offered on four consecutive days; in Massachusetts, the six workshops were 
offered over a period of between nine and twelve weeks. This longer period in Massachusetts 
provided participants more time to drop out of the demonstration. 

The higher completion rate in Washington than in Massachusetts may also reflect the 
incentive of the lump-sum payment. That is, in Washington, one of the five milestones for 
receiving a lump-sum payment was the completion of all the training modules. This incentive 
may, therefore, have inspired some people to complete the training module sequence. In 
Massachusetts, there was no comparable incentive since there was no lump-sum payment. 

Individual counseling sessions were attended more frequently and for a longer time 
among Massachusetts participants. Only 7 percent of Massachusetts participants did not receive 
any individual counseling, while the same was true for nearly 30 percent of Washington 
participants. The average number of counseling hours was 7.5 in Massachusetts compared with 
1 .5 hours in Washington. This difference may again be attributed to the scheduling of business 
training services in the two demonstrations. 

In Massachusetts, participants attended the Enterprise Seminar, followed by six 
workshops. The larger number of sessions in Massachusetts, combined with the longer elapsed 
time over which these sessions were offered, provided Massachusetts participants more contact 
with their counselors and thus more opportunities to schedule an individual counseling session. 
The ongoing contact with the counselors over a longer period of time ensured that the counselors 



ERIC 



64 



88 



Chapter 5 • Comparison of Demonstration Implementation 



were aware of the participants' progress through each step and thus could schedule counseling 
sessions when it became apparent that problems might be occurring. 

In Washington, the four training modules were scheduled to be completed within a one- 
week period. In such a short and intense period, participants had little opportunity to schedule 
a private session with their counselor prior to the completion of the training program. In many 
cases, participants did not have a scheduled session with their counselors until their milestone 
review. 

Participant S.\tisf action wtth Business Services 

In the follov/-up survey we asked demonstration participants their opinions of the 
demonstration business assistance services. Table 5.4 presents information on the opmions of 
demonstration treatment group members regarding the quality of the services they received. As 
shown in the table, opinions about the business training sessions were favorable and quite similar 
in the two demonstrations. Approximately four-fifths of those attending at least one training 
session rated the sessions as good or excellent. A slightly high/ir percentage rated their 
instructors as good or excellent. 

As noted previously, a higher percentage of Massachusetts treatment group members 
attended mdividual counseUng sessions. In both demonstrations, however, participants rated 
their counseling sessions and their counselors quite favorably. Again, approximately four-fifths 
had a favorable reaction. 

Business Starts by Participants 

In Washington, 333 businesses were started by treatment group survey respondents in the 
first 21 months after random assignemnt. In Massachusetts, 117 businesses were started by 
treatment group survey respondents during the first 19 months.^" Based on the number of 
treatment group survey respondents in each demonstration, on average, each treatment group 
member started .55 businesses in both Washington and Massachusetts. 



»The Massachusetts results are based on Cohorts 1 and 2 only. 
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Table 5.4 

Washington/Massachusetts Comparison 
Participants' Rating of Program Experiences 



Program Experieawss 


Washington 


Massadhus^ 


Business Training Modules* 






Number who attended at least one business 
training workshop/Enterprise Seminar or 
module 


508 


404 


Percentage who rated the business training 
workshops excellent or good 


80.1% 


79.2% 


Percentage who rated the business training 
workshop instructors excellent or good 


84.1% 


84.4% 


Individual Counseling Sessions* 






Number who reported attending counseling 
sessions 


268 


374 


Percentage who rated the sessions excellent or 
good 


83.6% 


78.9% 


Percentage who raied the business counselors 
excellent or good 


85.5% 


82.4% 



Source: PTS and Wave I Survey Data 

Tht responses presented are for individuals who reported attending the activity on the follow-up survey and who 
also had a PTS record indicating attendance at that activity 



Table 5.5 presents the characteristics of the businesses started by treatment group 
members. For example, the table presents the distribution of industries in wtiich businesses were 
started, whether the businesses were home-based, the structure of the enterprises, the percentage 
of the businesses owned by the treatment group member, and type of fmancing used. The 
results indicate that in both demonstrations, the service industry is the most common industry 
group for starting a business. Some examples of these businesses include personal services, 
accounting, and printing. Moreover, a substantial majority of all businesses started by treatment 
giup members were home-based; most businesses were sole proprietorship; and most required 
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the owner to invest his/her own money. We also examined the charateristics of businesses 
started by control group members (results not shown in table) and found these results to be quite 
similar to the above findings. 

Table 5.5 

Massachusetts and Washington 
Characteristics of Treatment Group Members' 
Self-Employment Enterprises 
(Percent) 





1 

Massachtisetts 1 
1 


Washington 


Industry 






Construction 


3% 


4% 


Manufacturing 


6% 


8% 


Wholesale/Retail 


.7% 


5% 


ocrviccs 




19% 


Other 


3% 


5% 


No Self-Employment 


58% 


60% 


Whether Operated From Home 






Home-Base 


32% 


29% 


Not Home-Based 


10% 


11% 


No Self-Employment 


58% 


60% 


Structure of Enterprise 






Sole Proprietorship 


34% 


33% 


Partnership 


1 % 


2% 


Corporation 


5% 


5% 


No Self-Employment 


58% 


60% 


Percent of Enterprise Owned 






Own 100% 


37% 


34% 


Share Ownership 


3% 


6% 


No Self-Employment 


58% 


60% 


Financial Investment 






Invested Own Money 


38% 


37% 


Did Not Invest 


4% 


3% 


No Self-Employment 


58% 


60% 



Source: Wave I Survey Data 
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The analysis in this report is based on data from the following sources: the Participant 
Tracking System (PTS), administrative records, telephone follow-up surveys and demonstration 
expenditure data. A detailed description of these data was presented in Benus, et al., 1993. 
Here, we present a brief description of the data sources that are used in the present analysis. 
First we present a description of the Participant Tracking System and its use in the analysis, 
followed by a description of the administrative data used in the evaluation. We then describe 
the two foUowup surveys conducted for this study^ 

Participant Trackeng System 

The Participant Tracking System (PTS) is an on-line database system developed by DOL 
to provide ongoing information about project participants and project services. The PTS served 
as an integral component of the demonstrations by performing such functions as targeti^ig project 
participants, generating letters to participants, randomly assigning individuals to treatment and 
control groups, and maintaining on-line information about project services. In addition to 
performing all these functions during the project implementation phase, the PTS also provided 
data for analyzing program operations. 

For the present analysis, the PTS provides data on: (1) individual characteristics; (2) 
demonstration services; and (3) UI payment information. Data on sample members' 
characteristics are used to describe claimants targeted for the demonstration. These data are 
also used to compare the characteristics of the treatment and control groups members. Finally, 
individual characteristics data are used in the impact analysis as control variables in multivariate 
regressions. 
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Data on demonstration services are used in the process analyses of the Washington and 
Massachusetts demonstrations, presented above in Chapters 3, 4, and 5. For example, we used 
data on participation in demonstration activities to assess the timing and level of participation 
in key demonstration activities. PTS data on sample members' UI benefit receipt were used to 
develop UI outcome measures such as total weeks paid, total benefits received, and whether the 
claimant exhausted benefits. 

ADMirVISTRATTVE RECORDS 

UI Wage Recc rds 

A second source of administrative records, available in Washington only, were State UI 
Wage Records that are reported by employers on a quarterly basis. For the evaluation, we 
obtained quarterly information on the total wages and hours worked of employees in covered 
employment during both the immediate pre-program and post-program periods. In particular, 
we obtained information on ''/ages and hours worked in covered employment for the five, full 
calendar quarters before the individual filed a new claim and entered the demonstration, for the 
quarter the claim was filed, and for the following nine calendar quarters. These data were used 
to construct key outcome measures of employment and earnings experiences, as well as control 
variables. In addition to providing an important source of supplemental employment and 
earnings outcomes information, the UI Wage Records data also enabled us to determine the 
Standard Industrial Classification (SIC) code of the employer, as well as whether the claimant 
returned to work for the same employer after the demonstration." 



" Approximately 10 percent of the quarterly records had missing data on hours worked. In these cases, h;. irs 
worked were imputed following a two-step procedure. For claimants with missing data in one (or more) of the five 
pre-experimental quarters, the person's average (real) hourly wage rate was computed during the entire pre- 
experimental period and applied to the total wages data in the specific period to impute hours worked for that 
quarter. A similar procedure was used for the nine post-enrollment quarters. For the few claimants for whom we 
could not compute an average wage (i.e., always had missing hours worked data), we estimated an hourly wage 
regression equation and applied the pn^dictcd values to total wages in the quarter to impute hours worked. The 
predictor variables for the hourly wage imputation equation included demographic characteriotics (e.g., age, age- 
squared, male dummy, education dummy variables, race/ethnicity dummy variables), site dummies, dummies for 
pre-program industry and occupation, and quarter and year dummy variables. 

" Using the employer tax account number for wages reported in each quarter we were able to determine 
whether claimants returned to work for the same employer and calculate how many quarters they worked for 
different employers. 
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UI Wage Records have several potential advantages over survey data for measuring 
earnings received from wage and salary employment. For example, UI records are not subject 
to interviewer bias or respondent recaU error. Also, these data are not subject to problems that 
arise from some respondents reporting net (after-tax) earnings, and others reporting gross 
(before-tax) earnings. Moreover, they are not affected by response-rate problems. 

Although the use of UI Wage Records has a number of advantages for the evaluation, 
it must be recognized that they do not include all earnings received by claimants.^^ In 
particular, they do not include wages in uncovered employment, earnings from self-employment, 
or wages earned in other states. In general, self-employed individ-ials constitute the large 
majority of uncovered employment. Other types of uncovered employment include casual labor, 
religious organizations, newspaper deliverers, insurance agents and real estate agents paid on a 
commission basis only, and barbers/hairdressers. 

A more important limitation of using UI Wage Records to develop measures of earnings 
from wage and salary employment arises for individuals who live near state borders. Because 
the system is state-based, it is impossible to distinguish individuals who work across the border 
in a different state from individuals who do not wotk in covered employment. This is 
particularly problematic for claimants in the Vancouver site ~ who comprise roughly one-quarter 
of the SEED analysis sample - and who generally live within a few minutes of the Oregon 
border and the Portland metropolitan area. To the extent that treatment and control group 
members find jobs in Oregon, this measurement error problem would bias the net impact of the 
program on UI wages toward zero. 

Because of the advantages of survey data for measuring earnings for all claimants 
(regardless of their geographical location), as well as in developing more precise measures of 
pre- and post-SEED wages, the main results presented in Chapter 7 rely on the interview data 
described later in this chapter. However, we also present an alternative set of impa'-t /esults 
based on outcome measures developed from ^JI Wage Records to determine how sensitive the 
results are to different measures. 
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" Moreover, because they are only available on a quarterly ba.sis, it is difficult to devebp precise measures 
of earnings for the pre- and post-SEED periods. This could be a particular problem if most of the effects of the 
deir.onstration were to occur relatively early in the claim spell. 
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Department of Revenue Data 

The third major source of administrative records for the impact evaluation is Department 
of Revenue (DOR) data (available only in Washington). For the evaluation, we obtained DOR 
records for individuals in the SEED Demonstration for an 11-year period, 1981-1992. This 
enabled us to develop several pre-program and post-program indicators of business activity. 
Below we describe the data obtained, their advantages and potential limitations. 

To obtain the DOR data, a file of all demonstration participants - both treatment and 
control group members ~ was prepared by the SEED unit and sent to DOR for matching 
purposes. The file contained the person's name, SSN, and when known, the person's Uniform 
Business Identifier (UBI). The UBI is a unique identification number that is attached to the 
person's business license. For businesses contained in the DOR file that were matched by DOR 
to demonstration participants - treatment group and control group members - we obtained the 
following data: 

• Date business opened and current status (e.g., whether open/closed); 

• Industrial sector; 

• Gross income and sales: 

• State and local sales taxes paid; and, 

• Business and occupation taxes paid. 

Because some businesses are required to pay taxes on a monthly basis, while others pay taxes 
on a quarterly or annual basis, we aggregated the sales and tax data into annual measures for 
all businesses. We also developed indicators of whether the business was "active" during a year, 
based on it being open during the year and having positive gross sales or income. 

The availability of DOR records provide a valuable supplement to the self-e nployment 
infomiation obtained from the follow-up interviews for claimants in Washington State. Of 
particular importance, they enable one to estimate the impact of the demonstration on State taxes 
paid, which is a component of the overall benefit-cost assessment. On the other hand, as 
described below, there are some important limitations to the DOR data that must be recognized. 
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First, similar to the UI Wage Records problem described above, because the DOR data 
ar; state-based, they only include business acti/ity in Washington State. Second, because the 
DOR data are a direct result of businesses paying various excise taxes on sales and income, and 
because individuals involved in fee-for- service businesses who receive less that $12,000 annually 
do not have to pay business and occupation taxes, it is likely that some of this income is 
unreported. Third, because the data relate to a specific DOR business registration number, and 
activity from more than one business can be reported under the same tax registration number, 
it is not generally possible to separate the business activity of a particular individual from that 
of his or her spouse, parents, siblings, or friends that are also reported on the same form. This 
makes it difficult to link the sales and tax information to specific treatment and control group 
members. Moreover, to the extent that SEED participants join businesses with pre-existing tax 
registration numbers, it greatly limits the usefulness of other data such as business opening and 
closing dates and industry'." 

A final concern about the DOR data involves the way in which the DOR records were 
matched to SEED participants, which could in turn result in potential differences in data quality 
between treatment and control groups. SEED treatment group members who started a business 
and received a lump-sum payment had to demonstrate they had a business license. Because the 
UBI associated with the license is the primary mechanism used by DOR to match businesses, 
it is possible that the matching process was biased in favor of obtaining relatively more DOR 
records for treatment group members than for control group members. That is, although the 
DOR file contains the SSN and name of the person listed as the business owner that can also be 
used for matching purposes, these data are often of lower quality and could potentially result in 
fewer matches. However, available evidence indicates that this was not a serious problem for 
the evaluation." 



■* In addition, information in the DOR data base concerning the date a business closed is not likely to be very 
reliable since business owners may simply choose to report "no activity" on their tax forms for several periods, 
rather than indicate their business closed. 

" First, the DOR data indicate no significant differences between treatment and control group members in any 
pre-program business activity measures. Moreover, based on an earlier pilot study, among individuals with known 
UBI. we were able to match 85 percent of the businesses based on SSN and name only. Thus, the extent of bias 
in these data between the two groups is likely to be small. 
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Survey Data 

Two followup surveys were conducted by Abt Associates' Survey Research Group using 
Computer Assisted Telephone Interviewing (CATI). The first of these follow-up surveys 
(Wave I) was administered to all Washington and Massachusetts demonstration participants 
(treatment and control group members) approximately 1.5 to 2 years after random assignment. 
The second survey (Wave 11) was administered approximately one year later to all Washington 
and Massachusetts Cohort 1 and Cohort 2 participants who responded to the Wave I survey. 
Only one followup survey (Wave I) was administered to the Massachusetts Cohort 3 participants. 
The Wave I survey for Cohort 3 was administered to a total of 496 participants^^ those 
randomly assigned from March 1, 1992 through January 19, 1993. 

The elapsed time between random assignment and the Wave I surveys is presented in 
Table 6.1. As seen in the table, the median survey follow-up period was 18.8 months in 
Massachusetts as compared with 21.3 months in Washington. Table 6.2 shows the median 
length of time between the Wave I and Wave n surveys is similar in both demonstrations (11.8 
months in Massachusetts and 11.4 months in Washington). Combining both periods, the total 
observation period in Massachusetts is approximately 31 months and the total in Washington is 
approximately 33 months. 

The administration of the follow-up surveys was complicated by the fact that the 
Massachusetts demonstration was implemented in three phases and over a three-year period. 
That is, to ensure a consistent follow-up period for each of the cohorts, the follow-up surveys 
had to be staggered. The staggered interview periods are depicted in Exhibit 6.1. As shown 
in the exhibit. Wave I interviews for the Washington sample were between January and April, 
1992. Wave I interviews for the three Massachusetts cohorts were administered as followb: for 
Cohort 1, between January and April, 1992; for Cohort 2, between January and April, 1993; 
and for Cohort 3, between January and March, 1994. 

Wave n interviews were administered between January and April, 1993 for the 
Washington sample. For Massacnu,.?tts Cohort 1, Wave n interviews were also conducted 



As discussed in Chapter 4, a total of 701 individuals were randomly assigned during the 1992-1993 Cohort. 
The analysis sample for this cohort, however, is the 496 with whom we attempted Wave I interviews. 
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Table 6.1 
Survey Follow-up Period 
Elapsed Time Between Random Assignment and First Follow-up Interview 

(In Months) 



Survey Follow-up Period 


xviassacnusews i 
(N=854) 1 


wasningion 
1 (N=1204) 


Minimum 


12.5 


17.8 


Lowest Quartile 


17.4 


20.3 


Median 


18.8 


21.3 


Third Quartile 


20.4 


22.4 


Maximum 


24.2 


30.6 


Source: Followup Survey 






Table 6.2 
Survey Follow-up Period 
Elapsed Time Between First and Second Follow-up Interview 

(In Months) 




Massachusetts 
(N=398) 


Washington 
(N=1087) 


Minimum 


8.9 


8.9 


Lowest Quartile 


11.6 


10.8 


Median 


11.8 


11.4 


Third Quartile 


12.1 


12.0 


Maximum 


13.5 


13.9 



Source: Follow-up Survey 
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Timing of Wave I and Wave II Surveys 
in Washington and Massachusetts 



Massachusetts 



Cohort 1 



Cohort 2 



Cohort 3 



RA 



Wave I 



Wave II 




I I I I I I I I I I I i I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I 
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between January and April of 1993. For Cohort 2, however, the interviews were conducted 
between January and April of 1994. There are no Wave n interviews for Cohort 3. The 
analyses of demonstration outcomes presented in subsequent chapters of this report are based on 
survey data fron. both Wave I and Wave n interviews. 

Contents of the Survey Data 

The follow-up surveys collected detailed pre- and post-program information about 
employment and earnings. The surveys also collected information on periods of unemployment, 
periods of time spent looking for work, demographic characteristics, and experiences with the 
programs. Specific categories of variables that were collected in the surveys are: 

Current and Most Recent Spells of Wage and Salary Employment 

• Length of each spell 

• Average number of hours worked each week 

• Salary for each spell 

• Industry and Occupation 

• Fringe Benefits received and personal satisfaction with the job 

Other Spells of Wage and Salary Employment in the Followup Period 

• Length of each additional spell of employment 

• Salary for each additional spell 

• Average number of hours worked each week 

All Spells of Self-Employment in the Followup Period 

• Industry, product, and organizational structure of the business 

• Hours worked per week 

• Participation of family members in ownership/operation of 
business 

• Sources of capital used to operate the business 

• Business liabilities and assets 

• Earnings from the business 
■ Fringe benefits received 

• Personal satisfaction with self-employment 

• Number of family members and other individuals employed in the 
business 
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Employment Prior to Random Assignment 

• Length of longest period of employment since age 18 

• Type of employment for that spell (wage and salary or self- 
employment) 

• Number of hours worked per week and salary for that employment 
spell 

Spells of Unemployment During the Followup Period 

• Number of spells of unemployment 

• Length of spells 

Periods of Time Spent Unemployed and Looking for Work 

• Amount of time per week spent looking for work 

• Type of job sought and desired salary 



Background Characteristics 

• Primaiy occupation 

• Marital Status 

• HousehoH size, family income and assets 



Participation in the Demonstration 

• Participation in demonstration activities and opinions about the 
quality of those activities 

• Reasons for not opening a business 

• Opinions about the most useful components of the program 

• Receipt of additional self-employment training other than SEED 

Response Rates and Characteristics of Survey Respondents 

Trained mterviewers attempted to interview all sample members (i.e., individuals who 
were randomly assigned). It should be noted that in Massachusetts, we did not attempt to 
interview all of Cohort 3. Due to budget limitations, we limited the interview sample to those 
sample members who were randomly assigned before January 19, 1993 (there were a total of 
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Table 6.3 
Washington and Massachusetts 
Survey Response Rates 
Wave I and Wave II 





AOitfl tNUupie 


Wave I 


Waven 






Respondents 


Respondents 






N 


% 


N 


% 


Washington 


1507 


1204 


79.9% 


1087 


90.3% 


Massachusetts 












Cohort 1 (1990) 


207 


177 


85.5% 


159 


89.8% 


Cohort 2 (1991) 


314 


272 


86.6% 


239 


87.9% 


Cohort 3 (1992-1993) 


496 


405 


81.7% 


NA 


NA 



Source: Followup Survey 



496 such individuals). In Table 6.3 we present the response rates for both Washington and 
Massachusetts and for each of the waves. 

In the Washington demonstration, interviews were attempted with a total of 1507 sample 
members (755 treatments and 752 controls). Interviews were completed with 1204 (604 
treatment group members and 600 control group members), for a response rate of 80 percent. 
For the Wave n survey, we attempted to interview all of the respondents to Wave I; 1087 (561 
treatments and 526 controls) responded to Wave n for a response rate of 90 percent. 

In the Massachusetts demonstration, 207 Cohort 1 participants (105 treatment group 
members and 102 controls) were called for the Wave I survey. A total of 177 Wave I interviewt 
were completed (89 treatments and 88 controls), for a response rate of 86 percent. For the 
Wave n survey, 159 interviews (82 treatments and 77 controls) were completed, for a response 
rate of 90 percent. 

For Cohort 2 in Massachusetts, a total of 314 Wave I interviews were attempted (158 
treatments and 156 controls). We completed 272 Wave I interviews (140 treatments and 132 
controls) for a response rate of 87 percent. We also completed 239 Wave n interviews (122 
treatments and 117 controls) for a response rate of 88 percent. 
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For Cohort 3, Wave I interviews were attempted with 496 individuals randomly assigned 
from March, 1992 through January, 1993 (250 treatments and 246 controls). A total of 405 
Wave I interviews were completed (213 treatments and 192 controls), for a response rate of 82 
percent. 

To assess whether survey respondents differed systematically from the total sample, we 
compare in Table 6.4 the demographic characteristics of the Wave I respondents with the total 
sample." As indicated by these results, the respondent sample is similar to the total sample 
in both Washington and Massachusetts. We found no statistically significant differences between 
respondents and non- respondents in either the Washington or the Massachusetts samples. We 
also examined the demographic characteristics of Wave n respondents (not presented here) and 
found no statistically significant response bias. 

In addition to assessing the effect of non-response on the representativeness of the two 
samples, we also compared the demographic characteristics of the Washington and Massachusetts 
samples. As indicated in Table 6.4, we found some statistically significant differences between 
the Washington and Massachusetts samples. The Washington sample has a higher percentage 
of individuals under age 25 (4 percent in Washington compared with 2.5 percent in 
Massachusetts). The Massachusetts sample has higher educational attainment, with 44.8 percent 
of the sample college graduates compared with 28.7 percent of the Washington sample. 
Massachusetts sample members are more likely to have previously worked in 
professional/technical/managerial occupations and Washington sample members are more likely 
to have worked in clerical occupations. Finally, the Massachusetts sample has a higher 
percentage of African Americans than does the Washington sample. 



" We did a similar comparison for Waves I and 11 and found very similar results. 
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Table 6.4 
Characteristics of 
Individuals Randomly Assigned and AU Survey Respondents 



\<rlUU aVid 


Massachusetts 


Washington 


Total 
Sample 
(N=»1222) 


Wave I 
Survey 
Respondents 
(N=854) 


Total 
Sample 
(N=1507) 


Wave I 
Survey 
Respondents 
(N=1204) 


Gender 












69.0% 


69.7% 


67.3% 


65.0% 


Age at Random Assignment 










Mean Age (in years) 


40.7 


40.9 


39.5 


39.7 


Percent Age < 25 ** 


2.5% 


2.0% 


4.0% 


3.3% 


Percent Age > =45 


27.9% 


29.3% 


27.9% 


28.3% 


Education 










Percent College Graduate *** 


44.8% 


46.7% 


28.7% 


29.8% 


Mean Education (in years) 


14.5 


14 6 


13.8 


n 9 


Prior Work Experience 










r rurcddiunai/ iccnniL'ai/ 


46.6% 


47.8% 


36.7% 


38.3% 


managerial occupation 










Clerical occupation *** 


10.0% 


9.9% 


13.1% 


14.0% 


Services sector 


28.5% 


27.5% 


27.7% 


28.2% 


UI Entitlement ($) 










Mean weeRiy beneiit 


$259 


$261 


$197 


$199 


Amount 










Mean Maximum Benefit 


$7723 


$7780 


$5427 


$5516 


Payable 










Race/Ethnicity 










Caucasian 


89.2% 


91.1% 


91.3% 


94.5% 


African American *** 


8.4% 


6.9% 


3.2% 


1.9% 


Hispanic 


1.5% 


.9% 


2.1% 


1.5% 


Otlier 


.9% 


1.1% 


3.5% 


3.0% 



Source: PTS 



Tests for differences between Massachusetts and Washington samples: 

*** Indicates values are signiiicandy different from zero at the .01 level. 
** Indicates values are significandy different from zero at the .05 level. 
* Indicates values are significandy different from zero at the .10 level. 
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In the first part of the report we described the design and implementation of the UI Self- 
Employment Demonstrations and the data sources available for the analysis. With this chapter, 
we begin Part n, which focuses on the evaluation of demonstration program impacts on key self- 
employment and wage and salary job outcomes. Throughout the evaluation presented below, 
we measure program impacts by comparing the experiences of treatment group members with 
the experiences of control group members following random assignment. 

In this chapter we concentrate on the impacts of the Washington Self-Employment and 
Enterprise Development (SEED) Demonstration on the self-employment and wage and salary 
employment experiences of demonstration participants.^* Before beginning our presentation 
of program impacts, we first describe the procedures used to estimate program impacts. We 
then present the detailed results for each outcome measure, one measure at a time; readers who 
are not interested in all of the details may wish tc skip to the summary of the findings at the end 
of the chapter. 

We first describe the program impacts of SEED on self-employment and wage and salary 
employment measures based on foUowup survey data. We begin with an analysis of the impacts 
of SEED on several key measures of self-employment outcomes based on followup survey data, 
including the likelihood of becoming self-employed, total time in self-employment, the Ukelihood 
of being self-employed at the time of the followup sup/eys, and total earnings from self- 
employment. We then examine the impact of SEED on several similar measures for wage and 
salary employment. We then combine self-employment and wage and salary employment and 



Throughout the analysis we do not distinguish between self-employment and business ownership. Inasmuch 
as we do not know who among the self-employed of today may become the large business owner of tomorrow, we 
choose not to distinguish among individuals who are self-employed and those who operate businesses that employ 
others. Thus, in our analysis we combine self-employment and business ownership and refer to business ownership 
as self-employment and vice versa. Later in the chapter, however, wc examine the impacts of SEED on number 
of employees hired. 
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eamings and examine the impacts of SEED on total employment and earnings outcomes. This 
section concludes with an examination of the impact of SEED on job creation using survey data. 

This is followed by a detailed examination of SEED impacts on various outcome 
measures derived from administrative data sources available in Washington State. We begin by 
examining the key issue of SEED impacts on benefit payraents received during the benefit year 
of the demonstration and on subsequent benefit years. We then examine alternative measures 
of self-employment outcomes based on administrative data sources, including the likelihood of 
receiving business income, total gross business income received, whether the business was still 
open, and State business taxes paid. In addition, we examine the impacts of SEED on the 
number of business employees and on wage and salary eamings based on quarterly UI Wage 
Records. 

Following the presentation of the basic impact estimates on both survey data and 
administrative records, we describe how the impacts of SEED differ among key claimant 
subgroups. The chapter concludes with a summary description of the main fmdings from the 
impact analyses that are used in the benefit-cost analysis presented in Chapter 10. 

In Chapter 8 we present the main demonstration program impact results for the 
Massachusetts Enterprise Project. Because of limited availability of administrative data sources, 
most of the impact analyses for the Massachusetts Enterprise Project reported in Chapter 8 rely 
on the followup survey data. In Chapter 9, we conclude Part n with a comparison of the 
patterns of results from the two self-employment demonstrations and present additional analyses 
to better understand the sources of potential differences in the results. 



ESTIMATION OF DEMONSTRATION IMPACTS 



ERIC 



Our measure of demonstration impacts is the difference between treatment group 
outcomes and what would have happened in the absence of the demonstration, as measured by 
the outcomes of the control group. For any given outcome, an unbiased measure of 
demonstration impact is provided by a simple difference in treatment and control group means. 
We refer to this simple difference in outcome means as the unadjusted program impact. In the 
tables that follow, we present the simple difference in means in the column labelled Difference. 
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A more precise, and still unbiased, impact estimate can be obtained through multivariate 
analysis, using covariates to explain some of the variation in outcomes across the sample. By 
including a variable that captures treatment status (i.e., T=l if the claimant is in the treatment 
group and T=0 if the claimant is in the control group), we can obtain an unbiased estimate of 
the average impact of the demonstration on the outcome by using ordinary least squares (OLS). 
We refer to impact estimates obtained from such multivariate regression techniques as the 
logression-adjusted program impact.'' The regression-adjusted impacts are presented in the 
tables that follow in the last column labelled Impact. 

In both the unadjusted and adjusted program impact estimates, a standard t-test can be 
calculated to determine whether the estimated impact is significantly different from zero. A 
single asterisk (*) following a given impact indicates statistical significance at the 10% level; a 
double asterisk (**) indicates significan . ^ the 5% level; and a triple asterisk (***) indicates 
statistical significance at the 1% level. Impacts with no asterisks are not statistically 
significantly different from zero at the 10% level. Thus, only estimates that are significantly 
different from zero at the 10% level or better are treated as evidence of a real effect of the 
demonstrations. 

Some of the outcome variables we investigate are binary in nature. For example, a 
person either did or did not experience self-employment during the observation period; a person 
either was or was not self-employed at the time of a follow-up survey. When estimating 
program impacts on these types of variables, logistic regression is generally considered more 
appropriate than OLS regression. Although as a practical matter there is often only a small 



" In addition to a dummy variable for treatment status, all of the regression equations reported in this chapter 
included age, age-squared, unemployment rate in the claimant's county of residence during 1990, and dummies for 
the following variables: demonstration sites, quarter in which the claimant's benefit year started, male, white, 
completed college, prior job in professional, technical or managerial occupation, prior job in services sector, 
whether the claimant indicated s/he intended to return to work for prior employer on the SEED application, spouse 
employed, having children under the age of six, having prior work experience related to proposed business, haviag 
a business at time of SEED application, having been self-employed before SEED application (but not at the time 
of application), being a high wage earner (i.e., in the upper quartile) in the four complete quarters before filiag the 
Ul claim, and being a medium wage earner (i.e., in the two middle quartiles) in the four complete quarters before 
filing the Ul claim. 
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difference between the logistic and OLS regression impact estimates, we report the logistic 
estimates in the tables that examine program impacts on binary outcome variables.*" 

The following tables summarize the demonstration impacts for the Washington SEED 
Demonstration. The results are presented for two overlapping time periods. First, for each 
outcome measure, we present the demonstration impact based on data obtained from the Wave 
I survey, which on average was 21 months after random assignment. In the tables, we refer to 
these outcomes as the Wave I results. 

Although the Wave U survey was designed to collect an additional 12 months of data 
after Wave I, for analysis purposes we combined the data from the two surveys to form a 
foliowup observation period that was approximately 33 months after random assignment. In the 
tables, we refer to such results as Wave II, as they are based on the Wave n respondents and 
cover the entire 33-month observation period, even though they draw on data from both foliowup 
surveys. 

The tables presented below focus on the overall impacts of SEED, and do not provide 
information on the claimant characteristics that affect the outcome measures or on whether the 
impacts of SEED vary by key claimant characteristics. In Appendbc B, we present summary 
information on the effects of covariates that are statistically significant in each of the regression 
mode s. We also examined whether the impact of SEED varied by numerous claimant 
characteristics, including gender, race, site, business experience, and whether the individual 
intended to return to the previous employer at the time of application to SEED. At the end of 
the chapter, we describe the general patterns of the subgroup impacts; more detailed information 
concerning the specific impact estimates for key subgroups is presented in Appendix C. 

Self-Employment Impacts 

Impacts on Entering Self-Employment. As described eariier, the SEED Demonstration 
provided business training, counseling, and financial assistance to IH claimants who were both 



*° Because the coefficient estimates from the logistic regressions are not easily interpreted, we transform them 
into more intuitive figures. Specifically, we transform the treatment/control logit coefficient estimate into an estimate 
of the impact of the program on the probability of the outcome variable occurring. To perform this transformation, 
we use the logistic coefficient estimates along with each individual's characteristics to derive an estimated probability 
of the outcome variable occurring for each sample member. Next, we calculate the average estimated probability 
foi 'he treatment and the control groups. FinaJly, we take the difference between these two average probability 
estimates and use this result as the transformed logit impact estimates. 
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interested enough in pursuing self-employment to apply to SEED and who were then randomly 
selected into the treatment group. Given the strong interest in self-employment and the types 
of services provided to treatment group members, one would expect more treatment group 
members than control group members to enter self-employment during the observation period. 
The results in Table 7.1 indicate that, indeed, treatment group members were much more likely 
than control group members to enter self-employment. Specifically, within the 21 -month 
average observation period corresponding to the Wave I survey, 57 percent of the treatment 
group, as compared with 32 percent of the control group, were self-employed at some point 
following random assignment.*' This 25 percentage point difference in the likelihood of self- 
employment for the two groups is statistically significant at the .01 level and corresponds to 
about an 80 percent effect evaluated at the mean of the control group. 

As indicated in Table 7.1, by the time of the second foliowup survey, some additional 
treatment group members and control group members had a self-employment experience. Over 
the entire 33-month observation period, 41 percent of the control group, as compared to 63 
percent of the treatment group, had at least one self-employment spell. The simple difference 
estimate of 22 percentage points in the likelihood of having a self-employment experience is 
slightly lower over the entire 33-month observation period than during the 21 -month period of 
Wave I, but still corresponds to over a 50 percent effect evaluated at the mean of the control 
group. 



Table 7.1 

Self-Employment Experience Since Random Assignment 





Control 


Treatment 


DilTerence 


Impact 


Wave I 


32% 


57% 


+25%'" 


+25%'" 


Wave n 


41% 


63% 


+22%'" 


+22%'" 



The Wave I results reported here differ slightly from the Wave I findings presented in our earlier report 
(Benus, Johnson, and Wood 1993), although all of the main findings are qualitatively similar. The differences are 
due to the use of improved definitions of certain outcome measures and slight differences in the analysis samples. 
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The regression-adjusted impact estimates of the likelihood of entering self-employment 
reported in the last column of Table 7. 1 of 25 percentage points for Wave I and 22 percentage 
points for the entire 33-month observation period are quite similar to the simple-difference 
estimates reported in the third column."^ Thus, we can conclude that the SEED Demonstration 
had a large positive impact on the likelihood of becoming self-employed, which persisted 
throughout the obsei-vation period. 

Impacts on Time in Self -Employment. To analyze the impact of the SEED 
Demonstration on the time spent in self-employment, we use annualized measures of the number 
of months in self-employment following random assignment. Specifically, using information on 
start dates and end dates of self-employment spells from the followup surveys, we calculated 
annualized months of self-employment as the proportion of the period from random assignment 
to the interview date that an individual sample member was self-employed multiplied by 12. 

As shown in Table 7.2, control group members on average worked at the iate of 1 month 
per year in self-employment throughout the first and second followup observation periods. 
Consistent with the results described above, treatment group members spent more time in self- 
employment on average than controls. In particular, on an annualized basis, treatment group 
members spent 3.2 months per year in self-employment during the first 21 months after random 
assignment and slightly less (3.0 months per year) over the entire 33-month observation 
period."^ These mean values translate into simple-difference estimates of the impacts of SEED 
on annualized time in self-employment of 2.2 months per year for the 21-month period and 2.0 
months per year for the entire 33-month observation period. These point estimates are the same, 
regardless of whether the smiple-difference estimator or the regression-adjusted approach is 



The following covariates are estimated to significantly increase the likelihood of self-employment in both the 
Wave I and Wave 11 regression models: being a college graduate, having a business at the time of random 
assignment, having experience in the area of the proposed business, having a spouse who is employed, and having 
earnings in the highest quartile in the four quarters before filing the UI claim. For additional details on the effects 
of the covariates in the impact regression models, see Appendix B. 

*^ The lower rate of time in self-employment over the entire 33-month observation period aa compared to the 
21 -month period is partly due to some treatment group members deciding to no longer work in self-employment, 
as well as a few starting businesses veiy late and contributing lower than average values on this annualized rate 
measure. 
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used.** Thus, these results indicate that over the nearly three-year observation period, 
treatment group members worked approximately 6 months more in self-employment than 
controls, on average. 



Table 7.2 

Time in Self Employment Since Random Assigniuent 





Control 


Treatment 


Difference 


Impact 


Wave I 


1.0 


3.2 


+2.2"' 


+2.3'" 




months 


months 


months 


months 


Wavell 


1.0 


3.0 


+2.0'" 


+2.0'" 




months 


months 


months 


months 



The greater amounts of annualized time spent in self-employment among treatment group 
members is partly a result of treatment group members being more likely to have a self- 
employment spell. In addition, this result could arise if SEED expedited entry into self- 
employment, or if SEED businesses were more stable and less likely to fail. For example, we 
might expect treatment group members to start their businesses relatively quickly, as the 
monetary incentive (i.e., lump-sum payment) they received upon completion of the five business 
milestones was larger, the sooner the milestones were met. In addition, treatment businesses 
could be less likely to fail than businesses established by control group members if the 
demonstration business assistance services were very effective. 

To examine these issues, we estimated two sets of hazard rate models for treatment and 
control group members overall, ard for key subgroups. Among those who were not self- 
employed at random assignnient, the first set of models examined the transition time from 
random assignment to the beginning of the first self-employment spell, wherie the length of time 



** Several covariatea were sUtistically significant in both the Wave I and Wave II regression models for 
annualized time spent in self-employment, including having related work experience in the proposed business area 
(+), being in the middle two quartiles or in the top quartile of the earnings distribution in the year before filing the 
UI claim (+) and having an employed spouse (+). 
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for claimants who never started a business by the time of the survey is treated as censored."' 
The results of these models indicated that treatment group members were significantly more 
likely to start businesses sooner than controls. This is true over the first five months after 
random assignment, with the estimated difference in calculated hazard rates being greatest at 
about 6-8 weeks after random assignment. This is roughly consistent with the timing of the 
receipt of lump-sum payments described in Chapter 4. Beyond five months, however, the 
estimated hazard rates of starting a self-employment spell are quite low and virtually identical 
between treatments and controls. 

We also examined whether transition times to beginning a first self-employment spell 
differed by key subgroups, by estimating hazards rate models with interaction terms for gender, 
prior income level, and whether the claimant indicated on the SEED application that s/he 
intended to return to work with the previous employer. Consistent with results described earlier, 
women are significantly more likely to initiate a self-employment spell sooner than men. We 
also found that among individuals who indicated they expected to return to work on their SEED 
application, the estimated hazard rates for treatment and control group members become quite 
similar after about 45 days. That is, the impact on transition time to self-employment dissipates 
much quicker for those treatment group members who expected to return to work, and suggest 
that if they do not start a business soon after random assignment, they are no more likely to 
return to enter self-employment than control group members with similar expectations. Finally, 
the results indicate significant differences by prior income level, with treatment impacts on 
transition time significantly greater throughout the first five months for claimants in the highest 
quartile of the earnings distribution in the prior year, and smallest for those in the lowest 
quartile. 

The second set of hazard rate models was estimated over the subset of claimants who 
initiated a self-employment spell. These models examined the transition time from the beginning 
of the first self-employment spell to the end of a self-employment spell to shed light on the 
impact of SEED on the likelihood of business failure. Overall, we found that the estimated 



ITie transition probability model incorporated duration dependence through a piex:c-wise spline, as well as 
complete treatment interactions with duration. 
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hazard rates of the risk of ending a self-employment spell for treatment and control group 
members were quite similar. Although the pattern of estimated hazard rates is consistent with 
slightly lower business failure rates for SEED treatment, group members, the differences were 
not statistically different from one another. Moreover, because of the very small sample sizes 
of businesses available for tliis analysis by subgroup, it was difficult to obtain precise estimates 
of the transition probabilities for any subgroups and none of the interaction terms were 
statistically significantly different from zero. 

Impacts on Self-Employment at Time of the Followup Surveys. The results described 
above indicate that treatment group members were more likely than control group members to 
have a self-employment spell during the observation period, and that they spent more time self- 
employed than controls. Although these results confirm that the SEED Demonstration was 
successful in encouraging business startups, they do not necessarily indicate program success 
since many of the self-employment experiences may not have been successful (i.e., they may 
have ended up in early terminations). One better indicator of program success is the proportion 
of individuals remaining in self-employment at the time of the followup surveys. 

In Table 7.3, we present information on the proportion of treatment and control group 
members who were self-employed at the time of each followup survey. As this table indicates, 
37 percent of the treatment group were self-employed at the time of the Wave I survey, as 
compared to 21 percent of the control group. This corresponds to a 16 percentage point 
difference, which is about a 76 percent effect evaluated at the control group mean. By the time 
of the Wave n survey, the number of controls in self-employment had increased somewhat (to 
25 percent), while the proportion of treatment group members in self-employment had only 
increased slightly to 38 percent. As a result, the simple-difference estimator yields a 13 
percentage point increase in the likeL'hood of self-employment at the time of the Wave n 
followup survey, which corresponds to over a 50 percent effect of the control group mean and 
which is statistically significant. The regression-adjusted impacts for the likelihood of self- 
employment at the time of the followup surveys are similar to the simple-difference estimates 
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and provide (x>nfmning evidence that SEED increased the likelihood of long-term self- 
employment.** 



Table 7.3 

Self-Employment at Two Different Observation Times 





Control 


Treatment 


Difference 


Impact 


Wave I 


21% 


37% 


+ 16%'" 


+ 16%'" 


Wavell 


25% 


38% 


+13%'" 


+ 12%"' 



Impacts on Self-Employment Earnings. Measuring earnings for the self-employed is 
extremely difficult for a number of reasons. First, many new entrants into self-employment do 
not receive regular income payments from their enterprise. As a result, they may report zero 
earnings when, in fact, they have positive but irregular earnings. Furthermore, some self- 
employed individuals underreport their earnings to federal and state authorities (e.g. , to illegally 
reduce their tax liability). These same individuals are likely to underreport earnings in the 
survey. Still others may not yet have received any earnings from their enterprise (e.g., a 
contractor who will get paid when a construction job is complete) and, thus, report zero earnings 
in the survey.*^ For all these reasons, collecting accurate self-employment earnings data is 
extremely difficult. 

In Table 7.4, we present information for self-employment earnings obtained from the 
Wave I and Wave n surveys. Similar to the time in self-employment outcome measure 
described above, self-employment earnings are constructed as an annualized measure. 
Moreover, the earnings figures have been deflated to 1990 dollars using a CPI index. As shown 
in Table 7.4, annualized self-employment earnings for the entire control group over the Wave I 



^ Some of the same covariates that were statistically significant in the Wave I and n regression models for the 
likelihood of having a self-employment spell at some point were also significant in both models for the likelihood 
of being self-employed at the time of the followup surveys. These include dummy variables for being a college 
graduate and being in the top qiuutile of the earnings distribution in the year before filing the UI claim. 

Results presented in our earlier report confirm the difficulties in measuring self-employment eamings. For 
example, we found that a large number of self-employed claimants (nearly two-thirds (64.5%) of the respondents) 
reported zero earnings in the survey. 
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observation period averaged $695 in 1990 dollars.** In contrast, annualized self-employment 
earnings at Wave I for the treatment group averaged $2,382. The unadjusted impact of $1,687 
is significant at the .05 level and the regression-adjusted impact of $1,596 is significant at the 
.10 level.*' 

The findings over the entire 33-month observation period are also quite similar. In 
particular, the unadjusted annualized estimate is $1,774 and the regression-adjusted estimate is 
$1,675. Thus, we conclude that treatment group members earned significantly more per year 
from self-employment as a result of the program Over the entire 33-month observation 
period, this amounts to about $4,600, and it appears that this difference is quite stable and could 
persist into subsequent years at the same rate. 



Table 7.4 

Earnings from Self-Employment Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 
Waven 


$695 
$576 


$2,382 
$2,350 


-l-$l,687" 
-l-$l,774"' 


-l-$l,596' 
-l-$l,675" 



Wage and Salary Employment Impacts 

The results described above indicate that the SEED Demonstration had significant 
positive impacts on the self-employment experiences of program participants. Below, we 
continue our focus on employment and earnings impacts, but shift our attention to examme the 



** This average eaminga figure includes cases where self-employment earnings were reported to be zero. 

The following covariates are estimated to significantly increase self-en^loyment earnings in both the Wave 
I and WRve II regression models: having a business at the time of random assignment, having experience in the area 
of the proposed business, having a spouse who is employed, and having earnings in the highest quartile in the four 
quarters before filing the UI claim. 

* This analysis included one case (a treatment group member) who reported earning more than $500,000 over 
tlic 21-month observction period. The decision to include the case was made in part because the administrative 
records available in Washington State were unable to convincingly demonstrate that the survey data were incorrect. 
The estimated self-employment earnings impacts for Wave I (II) are $1,042 ($1,123) when this case is excluded; 
both effects are statistically significantly different from zero. 
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impacts of the demonstration on the wage and salary employment experiences of program 
participants. 

SEED could affect wage and salary outcomes for a number of reasons. For example, 
suppose that the demonstration assisted the more capable treatment group members in pursuing 
self-employment; less capable treatment group members would likely remain unemployed or be 
employed in wage and salary jobs. In this instance, we would find a negative program impact 
of SEED on wage and salary earnings. 

Similarly, if the demonstration delayed entry into wage and salary employment for some 
treatment group members, their wage and salary earnings would likely be lower than they 
otherwise would have been. For example, if some treatment group members participated in the 
business training and began developing a business plan for several weeks before recognizing that 
self-employment was not for them, they are likely to have postponed their search for wage and 
salary employment during this interval compared to control group members who were not 
delayed in their search for wage and salary employment. A comparison of wage and salary 
earnings of all treatment and control group members would also yield a negative program impact 
for this reason. 

On the other hand, if the demonstration services (e.g., business counseling, peer group 
meetings) provided treatment group members with increased awareness of their marketable skills 
and enhanced their self-confidence and employability, SEED could have a positive effect on 
wage and salary earnings. Thus, even though SEED was designed to enhance the self- 
employment outcomes of program participants, it is likely to have significant effects on the wage 
and salary experiences of its participants. It is important to examine these effects on wage and 
salary outcomes to get a sense of the total impact of the program on employment and earnings 
outcomes, which is the summary measure used in the benefit-cost analysis. We now turn to an 
analysis of the impacts of SEED on wage and salary employment and earnings outcome 
measures. 

Impact on Likelihood of Wage and Salary Employment Experience. In Table 7.5 we 
examine the impacts of SEED on the likelihood of having a wage and salary employment 
experience during the observation period. These results indicate that a smaller proportion of the 
treatment group than the control group had at least one wage and salary job during the 
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observation period. By the time of the Wave I survey, 73 percent of the treatment group and 
80 percent of controls had at least one wage and salary job, and the difference between the 
groups is statistically sif.Tiificant at the .01 level. Moreover, the results in the second row 
indicate that over the entire 33-month observation period, 87 percent of control group members 
had at least one wage and salary job as compared to 81 percent of the treatment group. This 
difference of -6 percentage points is also significant at the .01 level, indicating that SEED had 
a negative impact on the Ukelihood of wage and salary employment. 



Table 7.5 

Wage and Salary Employment Experience Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 


80% 


73% 


-7%"' 


-7%'" 


Waven 


87% 


81% 


-6%'" 


-6%"* 



In the fourth column of Table 7.5 we present the results of logit regression models for 
the probability of having a wage and salary job during the observation period. The same 
covariates that were used in the regression models for self-employment outcomes described 
above were used in these regressions.'^ These results confirm our earlier conclusion that the 
SEED Demonstration had a significant and negative impact on the likelihood of having a wage 
and salary job during the observation period. The regression-adjusted impact indicates that, on 
average, the demonstration decreased the likelihood of a wage and salary job for treatment group 
members by 7 percentage points during the 21 -month observation period and by 6 percentage 
points during the 33-month observation period. 

Impact on Time in Wage and Salary Jobs. In Table 7.6, we report the impacts of 
SEED on annualized months in wage and salary jobs during the observation period. As this 



" As reported in Appendix B, holding other variables constant, three factors have a statistically significant 
impact on the likelihood of having a wage and salary job in both observation periods: being a current business owner 
at the time of SEED application (-), whether the claimant expected to be recalled to his/her prior employer at the 
time of the SEED application (+), whether the claimant had work experience in the area of the proposed businew 
(-) and whether s/he had earnings in the top quartile in the year prior to filing the UI claim (-). 
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table indicates, during the first 21 months after random assignment, treatment group members 
worked 4.1 months per year in wage and salary jobs compared to 4.7 months per year for 
controls. The reduction of .6 months per year is statistically significant at the .05 level. 

The results are quite consistent over the entire 33-month observation period. In 
particular, the annualized rate of time spent in wage and salary jobs by treatment and control 
group members increased slightly, such that the reduction in annualized months in wage and 
salary jobs due to SEBD remained about the same (i.e., .7 months per year reduction in time 
in wage and salary jobs). Thus, over the entire 33-month period, we estimate that SEED 
reduced total time in wage and salary employment by about 2 months. The regression-adjusted 
impacts shown in the last column of Table 7.6 are quite similar to the simple-difference 
estimates and are also statistically significant at the .05 level.'' 



Table 7.6 

Time in Wage and Salary Employment Since Random Assignment 





Control 


Treatment 


Differ^ce 


Impact 


Wave I 


4.7 


4.1 


-0.6" 


-0.6" 




months 


months 


months 


month 


Waven 


5.2 


4.5 


-0.7" 


-0.7" 




months 


months 


months 


months 



Impact on Wage and Salary Employment at Time of the Followup Surveys. In 
Table 7.7, we provide information on the proportion of sample members who were employed 
in wage and salary jobs at the time of the two followup surveys. As this table indicates, at the 
Wave I followup survey, 60 percent of control group members and 53 percent of treatment 
group members were employed in wage and salary jobs. This difference of 7 percentage points 



" Covariates that were statistically significant in both the Wave I and Wave n regression models for time in 
wage and salary employment include whether the claimant was male, age and age-sqitared, and whether the claimant 
was in the two middle quartiloa of the prior eammgs distribution. Specifically, the resulU indicate that holding other 
factors constant, males spent less time in wage and salary enq>loyment than females, that the time spent in wage 
and salary employment increased with age but at a decreasmg rate, and that those with medium earnings were more 
likely to spend time in wage and salaiy employment than those in the lowest quartile of earnings. 
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is the same as the regression-adjusted estimate and both are statistically significant at the .05 
level. These results reinforce the earlier pattern of findings on the likelihood of having a wage 
and salary job in the initial 21-months after random assignment. That is, just as we found that 
treatment group members were less likely than control group members to have a wage and salary 
job in the 21-months after random assignment, these results indicate that treatment group 
members are less likely to have a wage and salary job at the time of the first followup survey. 



Table 7.7 

Wage and Salary Employment at Two Different Observation Times 





Control 


Treatm^ 


Diff^nce 


Impact 1 


Wave I 


60% 


53% 


-7%" 


-7%" 1 


Wave n 


61% 


57% 


-4% 


-3%* 1 



By the time of the Wave n survey, however, the difference in the likelihood of wage and 
salary employment is much reduced. Specifically, at Wave H, 57 percent of treatment group 
members were employed in wage and salary jobs, as compared to 61 percent of controls. 
Moreover, an important distinction is that the difference between treatment and control group 
members at Wave n was not statistically significant at conventional levels. As seen in 
Table 7.7, the regression-adjusted Wave n impact of -3 percentage points is statistically 
significant at the .10 level. 

Impact on Wage and Salary Earnings. Measuring wage and salary earnings from 
survey data is less complicated than measuring self-employment earnings. Generally, those in 
wage and salary employment are paid regularly, their earnings are reasonably stable from month 
to month, and they are more likely to know their earnings than the self-employed. For all these 
reasons, we had substantially fewer complications in accurately measuring wage and salary 
earnings than in measuring self-employment earnings. Consistent with our measure of self- 
employment earnings, earnings in wage and salary employment are expressed in annualized 
terms and measured in constant 1990 dollars. 
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The results for annualized earnings in wage and salary employment are shown in 
Ti ble 7.8. These results indicate that during the initial 21-month observation period, treatment 
gnmp members earned $8,029 per year on average as compared to $9,231 per year for control 
group members." This yields an unadjusted impact of -$1,202, which is not significant at 
conventional significance levels. However, after controlling for other factors that affect 
earnings, the regression-adjusted impact of -$1,407 is significasit at the .10 level. 



Table 7.8 

Earnings from Wage and Salary Employment Since Random Assignment 





Control 


Treatmrat 


DiiOFerence 


Impact 


Wave I 
Wavell 


$9,231 
$9,920 


$8,029 
$8,414 


-$1,202 
-$1,506' 


-$1,407* 
-$1,780" 



The results for the entire 33-month observation period are similar. As described in 
Table 7.8, average annualized earnings in wage and salary employment for the treatment group 
over the entire observation period was $8,414, as compared to $9,920 for the control group. 
This unadjusted difference of -$1,506 is statistically significant at the . 10 level. Moreover, after 
controlling for other factors that affect earnings, the regression-adjusted annualized impact of 
SEED on wage and salary earnings is -$1,780, which is statistically significant at the .05 level. 
Thus, we conclude that SEED had a negative impact on the annual earnings of participants from 
wage and salary jobs during the observation period. 

Total Employment and Earnings Impacts 

The results described above indicate that the SEED Demonstration had positive and 
statistically significant impacts on a number of self-employment outcomes, but negative impacts 



" This average annualized earnings figure includes some cases where wage and salary earnings were reported 
to be zero. To the extent that some of the zero responses are likely to be misreported, the above average earnings 
figure is likely to imderstate actual wage and salary earnings. However, this data problem is much less serious than 
for self-employment earnings. 
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on several wage and salary outcomes. The overall impacts of SEED on the combination of these 
two types of employment experiences are, therefore, ambiguous. In this section, we present our 
findings of the impact of the SEED Demonstration on combined measures of employment and 
earnings outcomes. 

Impacts on Likelihood of Employment. The results presented above indicate that 
SEED had a positive impact on the likelihood of having at least one self-employment experience 
in the observation period and a negative impact on the likelihood of having at least one wage and 
salary job. In Table 7.9, we examine the impacts of SEED on the likelihood of having at least 
one employment experience during the observation period, either self-employment or a wage and 
salary job. These results indicate that during the initial 21-month followup observation period, 
SEED had a significant and positive impact on the likelihood of employment. Specifically, 96 
percent of the treatment group was employed in either wage and salary or self-employment 
during this period, as compared to 92 percent of controls. The unadjusted and the regression- 
adjusted impact estimates of 4 percentage points are both statistically significant at the .01 level. 
Thus, it appears that SEED facilitated early reemployment in general by increasing the likelihood 
of having a self-employment experience in the initial 21-month observation period. 



Table 7.9 

Combined Employment Experiences Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 


92% 


96% 


+4%'" 


+4%"' 


Wave n 


97% 


98% 


+ 1% 


+ 1% 



Over the entire 33-month observation period, however, this short-term effect disappears 
as virtually all of the sample members had held a job of some type during the period. 
Specifically, 98 percent of the treatment group and 97 percent of the control group held a job 
or were self-employed at some point in the 33-month observation period. The difference is not 
statistically significant at conventional levels; the regression adjusted impact estimate is also 
insignificant. 
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Impacts on Combined Time in Wage and Salary Employment and Self-Employment. 
The results described earlier indicate that SEED significantly increased the amount of time per 
year spent in self-employment, but decreased the time per year in wage and salary jobs. To 
examine the overall effects of SEED on time in either wage and salary or self-employment, we 
created an annualized measure of comljined months employed in any type of job. The total 
measure is not simply the sum of the separate annualized measures of time in self-employment 
and time in wage and salary employment.** Rather, the combined measure takes into account 
information on the dates of each employment experience to avoid double-counting periods during 
which individuals are working at both self-employment and wage and salary employment. 

The results for annualized total time in employment are presented in Table 7.10. For 
the entire control group, the average annual time in either wage and salary employment or self- 
employment during the initial 21-month observation period was 5.9 months. Thus, control 
group members were employed at some job roughly one-half of the initial 21 -month observation 
period. This compares to 7. 1 months per year for the treatment group. The unadjusted impact 
of 1.2 months is statistically significant at the .01 level, as is the regression-adjusted impact 
estimate of 1.3 months. 



Table 7.10 

Total Time in Employment Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 


5.9 


7.1 


-f-1.2'" 


-HI. 3*" 




months 


months 


months 


months 


Wave n 


6.4 


7.5 


-Hl.r" 


-Hl.r" 




months 


months 


months 


months 



The results are similar over the entire 33-month observation period. The annualized time 
in employment increased for both the treatment and control groups - to 7.5 months and 6.4 



ERIC 



^ As a result, the overall impacts of SEED on this measure do not equal the simple sum of the impacts on time 
in self-employment and time m wage and salary employment. Moreover, the combined results are not equal to the 
sum of the separate impacts because of differences in sample size due to missing data on one or the other 
components. 
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months, respectively ~ resulting in a simple-difference estimate, of the impact of SEED of 1 . 1 
months per -"ear. The corresponding regression-adjusted estimate was 1 . 1 months per year, 
which was also statistically significant at the .01 level. Thus, we conclude that the SEED 
program unambiguously increased total time in employment by approximately one month per 
year, or nearly 3 months over the entire observation period. 

Impacts on Employment at Time of the FoUowup Surveys. The results of the previous 
sections indicate that SEED increased the likelihood of being self-employed at the time of the 
followup surveys, but slightly reduced the likelihood of being employed in a wage and salary 
job at the followup surveys. In Table 7. 11 , we examine the impact of SEED on the likelihood 
of being employed in either type of employment at the time of the followup survey. 



Table 7.11 

Employment at Two Different Observation.Times 





Control 


Treatment 


Difference 


Impact 


Wave I 


75% 


78% 


+3% 


+3% 


Wave n 


78% 


84% 


+6%" 


+5%" 



At the time of the Wave I survey, 75 percent of the control group and 78 percent of the 
treatment group were employed in either self-employment or a wage and salary job. This simple 
difference of 3 percentage points, and the regression-adjusted impact of 3 percentage points, are 
not statistically significant at conventional levels. 

This is in contrast to the findings at the end of the 33-month period. Specifically, 84 
percent of treatment group members were employed in either self-employment or wage and 
salary employment at the time of the Wave n survey, as compared to 78 percent of Dntrols. 
This simple difference of 6 percentage points was statistically significant at the .05 level. 
Controlling for other factors that also affect the likelihood of employment results in a regression- 
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adjusted impact of SEED of 6 percentage points, which is significant at the .05 level. Thus, it 
appears that SEED has long-run impacts in facilitating claimants' likelihood of reemployment." 

Impacts on Combined Employment at Time of the Followup Surveys. The analyses 
described above examine the impacts of SEED on the likelihood of self -employment or wage and 
salary employment at the time of the followup surveys. Although such analyses provide a 
valuable overall summary about the impacts of SEED on the likelihood of reemployment, they 
do not provide any information about the extent to which claimants are involved in both types 
of employment simultaneously.- To more fully understand the impacts of SEED and the 
experiences of claimants in pursuing self-employment, it is important to understand the extent 
to which individuals were only pursuing self-employment, only pursuing wage and salary 
employment, or whether they were employed in both activities simultaneously. In this section, 
we provide information on this issue. 

To address this issue, we divided employment status at the time of the followup surveys 
into four states: (1) not employed; (2) employed only in a wage and salary job; (3) self- 
employed only; and (4) employed in both wage and salary and self-employment. In Table 7. 12, 
we provide simple descriptive statistics on these four alternative employment states. As this 
table indicates, at the time of the Wave I followup survey, approximately 12 percent of the 
treatment group was employed in both wage and salary jobs and self-employed as compared to 
6 percent of the control group. In addition, about 25 percent of the treatment group was only 
self-employed at the time of the first survey, as compared to 15 percent of the control group. 
Thus, over one-third of the overall 15.6 percent difference in the likelihood of self-employment 
at Wave I was due to treatment group members being more likely to be employed in both wage 
and salary jobs and in self-employment. 



ERIC 



" Only the prior earnings covariates consistently affect the likelihood of being employed at both followup survey 
points. Specifically, claimants in the middle and top quartiles of the earnings distribution in the year before filing 
a claim are more likely to be employed at the time of the followup surveys than other individuals. 
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Table 7.12 

Combined Employment States at Different Observation Times 





Control 


Treatment 


DifTerence 


Wave I 








Not Employed 


25.2% 


21.7% 


-3.5% 


Employed Only in 
Wage and Salary 


53.5% 


41.4% 


-12.1% 


Self-Eraployed Only 


15.0% 


25.0% 


+ 10.0% 


Employed in Both 


6.3% 


11.9% 


+5.6% 


Wave n 








Not Employed 


21.9% 


15.9% 


-6.0% 


Employed Only hi 
Wage and Salary 


53.2% 


46.7% 


-6.5% 


Self-Employed Only 


16.9% 


26.6% 


+9.7% 


Employed in Both 


8.X)% 


10.9% 


+2.9% 



The results at the time of the Wave n survey are presented in the bottom four rows of 
Table 7.12. For the most part, the employmen* patterns at the time of the Wave n survey are 
quite similar to the Wave I patterns. For the control group, a similar proportion are employed 
only in wage and salary jobs, and the smaller overall number who are not employed are balanced 
by a similar increase in the proportion who are self-employed only or who are employed in both 
wage and salary employment and self-employment. Among treatment group members, the 
proportion in self-employment only and the proportion in both types of employment are quite 
similar to the Wave I levels; the reduction in the proportion not employed from Wave I to Wave 
n is primarily accounted for by an equal increase in the proportion mployed only in wage and 
salary jobs. 
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To determine the impact of SEED on this joint distribution of employment states, we use 
a multinomial logit model." The results of the multinomial logit model are summarized in 
Table 7.13. For ease of interpretation, in this table we report the program impacts on the 
estimated odds ratios of each employment state relative to not being employed at the time of the 
foUowup survey. An odds ratio is the simple ratio of two probabilities, and thus represents the 
relative likelihood of one alternative to another. In this application, we are examining the 
impacts of SEED on the relative likelihood of three different employment states to the alternative 
of not working at the time of the foUowup surveys. If the odds ratio is less than 1 , this indicates 
that SEED reduced the likelihood of the particular alternative relative to not working; if the ratio 
is greater than 1 , it indicates the program increased the relative likelihood of the alternative 
relative to not working. 

The results in Table 7. 13 are quite similar both qualitatively and quantitatively across the 
two followup surveys and indicate that SEED has a large and statistically significant impact on 
the odds in favor of self-employment only relative to not being employed, and of both self- 
employment and wage and salary employment relative to not being employed; the results indicate 
that SEED did not affect the likelihood of working in wage and salary employment only relative 
to not working at all. Moreover, the size of the estimated coefficients are reasonably similar 
across these two employment states and across survey waves. In particular, the odds in favor 
of being self-employed only relative to not working are about twice as high for treatment group 
members than for controls in both survey periods. The results also indicate that the estimated 
odds in favor of working both in wage and salary employment and self-employment are larger 
for treatment group members than for control group members in Wave I (2.4) than in Wave n 
(1 .9). This indicates a strong long-term positive impact of SEED on the likelihood of working 
in self-employment only (relative to not working) and a very large short-term effect on working 



The multinomial logit model for this application can be written as: 

log [P^j/PJ = Xibj + cTi , j =2,3,4; i = l N, 

where P. is the probability that employment status j is chosen by individual i; Xj are the same set of independent 
variables for individual i that were included in the impact regression models described earlier; T, is a treatment 
dummy variable representing the treatment status for individual i; and bj and c are coefficients to be estimated. To 
obtain identification, we normalize against the group who are not employed (status of j = 1) at the time of the survey 
and impose the constraint that b,=!0. Given this constraint, c can be interpreted as the impact of SEED on the 
logarithm of the ratio Pj/P,. That is, c is the impact of SEED on the logarithm of the odds in favor of being in 
employment status j relative to not being employed. 
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in both wage and salary and self-employment, which declines somewhat over time, although it 
is still reasonably large and statistically significant at Wave n. 



Table 7.13 

Multinomial Logit Estimates of the Impacts of SEED on the Odds of 
Combined Employment Status at the Time of the Followup Surveys 



Survey Wave and £knpIoyinent Status 


Impact on Odds Ratio 


Wave I 




Employed Only in Wage and Salary 


0.86 


Self-Employed Only 


1.9r" 


Employed in Both 


2.41"' 


Wave n 




Employed Only in Wage and Salary 


1.08 


Self-Employment Only 


2.12'" 


Employed in Both 


1.93"' 



Impacts on Total Annualized Earnings. The final results in this section concern total 
earnings over the observation period from both wage and salary jobs and self-employment. As 
shown in Table 7.14, average annualized combined earnings from both wage and salary 
employment and self-employment over the 21-month observation period for the control group 
was $10,499. Total annualized earnings over the same period for the treatment group averaged 
$1 1,477. The unadjusted annualized impact of $978 and the regression-adjusted impact of $566 
are not significantly different from zero at the . 10 level. 

The results for the entire 33-month observation period are broadly similar. Specifically, 
although the annual rates of earnings are slightly higher ~ $11,590 for controls and $12,585 for 
the treatment group ~ the simple-difference estimate of $995 per year is very similar to the 
Wave I estimate. The regression-adjusted impact estimate of $289 is considerably smaller. 
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although neither point estimate is significantly different from zero at the .10 level. Thus, we 
cannot conclude that SEED had any impact on annualized combined earnings from self- 
employment and wage and salary employment. 



Table 7.14 

Combined Earnings from Employment Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 
Waven 


$10,499 
$11,590 


$11,477 
$12,585 


+$978 
+$995 


+$566 
+$289 



Other Demonstration Impacts 

In addition to providing employment and earnings for the business owner, small 
businesses often generate wage and salary employment for others. Frequently, when small 
business owners need employees they hire famUy members. In addition, small businesses also 
generate employment for nonfamily members. In this section, we analyze the demonstration 
impacts on total nonparticipant employment, where nonpaiticipant employment includes all 
famUy and nonfamUy wage and salary employees of the business, excluding the demonstration 
participant. 

We measure the effect of the demonstration on nonparticipant employment as the 
difference between the mean number of employees per treatment group member and the mean 
number of employees per control group member. Since this analysis is conducted over all 
treatment and control group members (whether they have a business or not), those individuals 
without a business are assumed to have no employees. 

As indicated in Table 7.15, SEED did not significantly increase the employment of 
nonparticipants during the first 21-months of the demonstration, but did have significant impacts 
over the entire 33-month observation period. Specifically, at Wave H, treatment group members 
overall averaged .42 nonparticipant employees, as compared to .17 nonparticipant employees for 

" The much lower program intact estimate obtained from the regression model is surprising and requires 
further examination. 
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the control group overall. This difference of .25 nonparticipant employees per treatment group 
member is statistically significant at the .01 level. This is consistent with the large differences 
described above between treatment group members and conUols in the likelihood of starting a 
business, the time spent in self-employment, and differences in the earnings from self- 
employment. 



Table 7.15 

Job Creation Impacts per Members: Total Employment 





Control 


Treatment 


Difference 


Impact 


Wave I 


0.22 


0.30 


0.08 


0.07 


Waven 


0.17 


0.42 


0.25*** 


0.25*** 



The above job creation impacts may be more easily understood by converting the "per 
member" job creation impacts to total job creation impacts. ITie total number of jobs 
attributable to the SEED Demonstration may be derived as follows. Since control group 
members employ, on average, .17 nonparticipant employees (see Table 7.15), the entire control 
group, is estimated to employ approximately 128 nonparticipants (i.e., .17 jobs per control 
group member * 755 control group members in the demonstration). In contrast, the entire 
treatment group is estmiated to employ approxiiiiately 316 nonparticipants (i.e., .42 jobs per 
treatment group member * 752 control group members). The treatment/control group difference 
is 188 jobs (i.e., 316-128). The SEED Demonstration, therefore, created a total of 188 
nonparticipant jobs during Wave H. 

Impact Results Using Administrative Data 

The impact results described above examine various measures of self-employment and 
wage and salary employment and earnings using data from the two followup surveys. In the 
remainder of this chapter, we supplement these findings by providing estunates of the impact 
of SEED on several similar outcome measures developed from administrative data sources 
available in Washington State, as well as examine the program impacts on total benefits 



105 



Chapter 7 • Washington Demonstration Impacts 



received. Although we regard the impact results based on the survey data discussed above as 
the best estimates of SEED impacts on employment-related outcomes, the results described 
below provide valuable additional evidence concerning SEED impacts on similar outcome 
measures, as well as impacts on other important outcome measures not directly available through 
the followup surveys. 

It is important to note that the results reported below are presented in a somewhat 
different format than the employment and earnings impacts based on survey data described 
above. First, because the time periods for which the administrative data were available do not 
correspond to the times that the surveys were administered, we present the results below for 
somewhat different followup periods. Second, because the data were obtained for specific 
calendar periods (i.e., quarters, years), it was straightforward to estimate the impact of SEED 
on separate postprogram calendar periods. Thus, rather than combine the administrative records 
across multiple periods and then estimate annualized impacts from several periods of data, we 
instead estimate separate impacts for several distinct postprogram periods. This provides 
additional information concerning the tune pattern of program unpacts. For most measures, one 
can obtain an annualized estimate over the entire observation period similar in concept to the 
findings presented for Wave I and n above by summing the estimated impacts over the separate 
periods and converting it to an annual rate. 

The remainder of the chapter is organized as follows. First, we present our analysis of 
the impacts of SEED on UI benefit payments for the benefit year in which the claimants were 
enrolled in the demonstration, as well as for subsequent years. We then present average impacts 
of SEED on several self-employment outcome measures based on Washington State Department 
of Revenue (DOR) administrative records. The chapter concludes with an examination of UI 
Wage Records data to estimate the impacts of SEED on the employment and earnings of 
participants in covered wage and salary employment and on employees hired and wages paid to 
employees in the businesses started by treatment and control group members. 

SEED Impacts on UI Outcome Measures 

A central issue in the UI Self-Employment Demonstrations concerns the impacts of the 
program on the receipt of total benefits. In the interim report, we described in detail the impacts 
of SEED on several measures of UI benefit receipt during the benefit year corresponding to the 
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UI claim that lead to enrollment in SEED. Below we briefly summarize those results and 
provide new findings related to the receipt of UI benefits in subsequent years. 

Length of First UI Spell. Using data from the UI history files in the PTS, we 
developed a measure of the number of weeks of the first spell of UI receipt." As shown in 
Table 7.16, controls experienced first spells of UI benefits of 17.5 weeks on average. In 
contrast, the length of the first UI q)ell for treatment group riiembers was only 11.4 weeks. 
This leads to simple difference estimates and regression-adjusted impact estimates of a 6. 1 week 
reduction in first UI spell, which is statistically significant at the .01 level. The large reduction 
in length of first spell is primarily due to treatment group members completing their five 
milestones to receive their lump-sum payment and then no longer drawing regular UI payments. 



Table 7.16 
Length of First UI Spell (Weeks) 



Control 


Treatment 


DilFference 


Bnpact 


17.5 


11.4 


-6.1'" 


-6.1'" 



UI Benefit Payments Received. To examine both the immediate and total effects of 
SEED on UI payments during the benefit year, we calcuMed both the UI payments received 
during the first UI spell and the total UI payments received during the entiiie benefit year, with 
and without the lump-sum payment. As summarized in Table 7.17, these results indicate that 
controls received an average of $3,368 during the fu^t spell. Because controls received $3,792 
on average over the entire benefit year, it is clear that the large majority (nearly 90 percent) of 
the UI payments received during the year occurred during the first spell. The unadjusted and 
regression-adjusted impact estimates are very similar for both measures and indicate that, 
excluding the lump-sum payment, SEED had a significant impact in reducing the amount of UI 



* The first spell of UI benefits is conceptually defined as weeks of continuous receipt of UI payments, and does 
not include the waiting week. Moreover, in conceptually attempting to link the end of a spell with the beginning 
of a job, we do not consider a spell to have ended if there is a minimal break in receipt of payments after whidi 
respondents continue to receive UI benefito. Because Washington State uses a bi-weekly payment system, we 
decided to concatenate spells that had breaks of two weeks or less in payments and treat them as a single spell of 
length equal to the number of weeks with positive payments. In this way, we ensure that claimanto who receive 
a UI payment every two weeks have not ended their spell. 
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benefits received by claimants. For example, we estimate that SEED reduced the UI benefit 
payments received during the first spell by $1,270. Moreover, given the estimated program 
impact of -$1,448 over the entire benefit year, the results indicate that the large majority of the 
impact on UI benefits paid occurred during the first spell. 

As described in Chapter 3, a total of 451 treatment group members completed 
their required project milestones and obtained a lump-sum payment. The average lump-sum 
payment for these 451 claimants was $4,225. Although these lump-sum payments were made 
with federal research funds ~ and not from the UI Trust Fund - it is important to understand 
what the total impact would be if these payments were funded by the same funding source. As 
shown in the third row of Table 7.17, taking into account both UI benefit payments plus lump- 
sum payments, treatment group members are estimated to have received $1,098 more on average 
than controls. Thus, there would be a net cost in the short-term of about $1,000 per claimant 
for operating a self-employment program like SEED that included lump-sum payments. 



Table 7.17 

UI Benefit Payments Received Duiing Initial Benefit Year 



Time Frame 


Control 


Treatment 


Difference 


Impact 


First SpeU 


$3,368 


$2,086 


-$1,282'" 


-$1,270'" 


FuU Benefit 
Year 


$3,792 


$2,333 


-$1,459"' 


-1,448'" 


FuU Benefit 
Year Including 
Lump Sum 


$3,792 


$4,857 


$1,065"' 


$1,098'" 



A related issue concerns the extent to which SEED was able to simulate a policy of 
cashing out UI benefits. As described in Chapter 2, although the lump-sum component of the 
SEED Demonstration was intended to simulate a cash-out of UI benefits, because UI is an 
entitlement program and these benefits could not be denied for demonstration purposes, it was 
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not Strictly possible to test a cash-out policy. Operationally, this meant that participants could 
return to the regular UI program after receiving their lump-sum payment and draw the remainder 
of their UI entitlement in the form of bi-weekly payments provided they met the normal UI 
eligibility requirements, including the work search requirement. Our results indicate that, for 
the most part, SEED can be considered to have tested a cash-out policy. Specifically, only 6.9 
percent of the treatment group members who received a lump-sum payment (4. 1 percent of the 
entire treatment group) returned to UI after receiving the lump-sum payment to draw additional 
UI payments from their original claim. Among those who returned to UI, the average additional 
UI benefits received was $1,839. This corresponds to about $75 per claimant over the entire 
treatment group. 

Receipt of Subsequent UI Benefit Payments. In addition to examining impacts on UI 
payments received during the benefit year, it is also important to understand whether SEED 
affects subsequent UI benefit payments. To investigate this issue, we obtained data from SEED 
program staff on UI benefit payments received by all treatment and control group members 
through the first half of 1994. These results are summarized in Table 7.18. 

The first row of the table provides information on the proportion of demonstration sample 
members who file a subsequent claim for a new benefit year and receive at least one payment 
from the new claim. Our results indicate that 31.8 percent of all controls filed a claim for a new 
benefit year after SEED. This compares to 31.0 percent of all treatment group members, 
resulting in a very small difference that is not statistically significant. 

The second row reports the average amount of subsequent UI benefits received. As a 
group, controls received $1,296 in regular UI benefit payments on average from claims filed 
during the (roughly) three year period after SEED. This compares to $1,214 on average for 
treatment group members. The regression-adjusted impact estimate of -$85 is not statistically 
significantly different from zero. Thus, SEED did not affect receipt of subsequent Ui benefits. 

In the last row of Table 7.18 we report the impact of SEED on total UI payments 
received over both the initial UI claim and the subsequent claims observed, and including the 
lump- sum payment. As shown in the last column, our results indicate that the program 
increased the UI benefits received by $1,013. As described above, all of the estimated increase 
occurred during the initial benefit year. 
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Table 7.18 

Receipt of Subsequent UI Benefit Payments 



UI Measure 


Control 


Treatment 


Difference 


Impact 


UI Benefits After 
SEED 


31.8% 


31.0% 


-0.8% 


-0.8% 


Amount of UI 
Benefits Received 
After SEED 
Year 


$1,296 


$1,214 


-$82 


-$85 


Total UI Benefits 
Received 


$5,088 


$6,07i 


$983'" 


$1,013'" 



SEED Impacts on Self-Employment Outcome Measures 

The Department of Revenue (DOR) in the State of Washington maintains extensive 
information on individual businesses that operate in the State and these data were linked to 
individuals in the demonstration for evaluation purposes. This includes information on business 
activity and on state tax payments. Specifically, we obtained DOR information for three years - 
- 1990,*' 1991, and 1992 ~ for the following measures: whether the sample member received 
busmess income; gross business receipts; whether the business was still open; and sales taxes 
and business and occupation taxes paid. Similar to the format used for presenting the results 
based on survey data, below we present the mean values of the outcome measures for the control 
group, the treatment group, and the calculated unadjusted and regression-adjusted impacts of 
SEED for each measure.*^ 

Impacts on Whether Claimants' Received Business Income. In Table 7.19, we 
examine the impact of SEED on the likelihood of receiving business income based on DOR 



"Although the data for 1991 and 1992 arc clearly post-enrollment for ell demonstration claimants, the measures 
for 1990 could include business income that occurred before SEED enrollment for some claimants. 

* The covariates used in these regressions are the same as those used in the models for employment and 
earnings outcomes based on foUowup survey data. 
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records. As shown in the first column, the proportion of control group members who received 
some business income was relatively stable over the 1990-1992 period. Specifically, in 1990, 
13.3 percent of claimants in the control group had businesses that received some income/sales; 
in 1991, the rate increased sUghtiy to 14.8 percent; and by 1992, the proportion with business 
income declined slightly to 12.9%." As shown in the second column of Table 7.19, the 
pattern for treatment group members is somewhat different, with 27.7% of all treatment group 
members receiving some business income in 1990, and with this rate declining in each 
successive year to 23.3% in 1991 and 19.1% in 1992. 



Table 7.19 
Receipt of Business Income 



Year 


Control 


Treatment 


Difference 


Impact 1 


1990 


13.3% 


27.7% 


14.4%"* 


15.1%*" 1 


1991 


14.8% 


23.3% 


8.5%*" 


9.4%"' 


1992 


12.9% 


19.1% 


6.2%*" 


7.0%*" 1 



Consistent with the results based on survey data, both the unadjusted and regression- 
adjusted differences indicate that claimants in the treatment group are significantly more likely 
than controls to be active in business during each of the three years, as indicated by having 
received some business income. In particular, SEED appears to have a very large initial impact 
on starting a business, but the impact is somewhat less over the longer term. For example, the 
regression-adjusted impact results indicate that SEED had a 14.4 percentage point increase in 
the likelihood of having an active business in 1990, which corresponds to over a 100% effect 
evaluated at the mean of the control group. By 1992, the impact of SEED is estimated to be 7.0 



*' Because of updates in the administrative records, these results are slightly different than the results presented 
in our interim report (Benus, Johnson, and Wood, 1993). 
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percentage points, which is considerably lower, but still corresponds to a large percentage impact 
of over 50% of the control group mean.*^ 

Impacts on Whether Busmess Still Active/Open. In Table 7.20, we provide 
information on whether the businesses are still open according to DOR records. In the first 
column, we see that 21.4 percent of all controls had businesses that were still considered to be 
open at the time the DOR records were collected (July 1993) and nearly three-fourths of these 
open businesses (15.6 percent) had received some income following random assignment." In 
contrast, nearly 40 percent of all SEED treatment group members had a business that was still 
considered to be open, although only about two-thirds (26.6 percent) had received some business 
income in the three-year period. The estimated impact of SEED on both of these measures is 
large and statistically significant, indicating that SEED has a considerable impact on the 
likelihood of having an open and/or active business (70-85 percent effect, evaluated at the 
control group mean). 



Table 7.20 
Busmess Status (July 1993) 



Busuiess Status 


Control 


Treatment 


Difference 


Impact 


Still Open 

StiU Open and 
Received Some 
Business Ir^come 


21.4% 
15.6% 


39.3% 
26.6% 


17.9%*** 
11.0%*** 


19.1%*** 
12.1%*** 



" In our interim report, we decomposed the SEED businesses into those that were active in the first year only, 
the second year only, or both the first and second years, to provide information ou the timing of business startups 
and the rate of business failures. TTiese results indicated that roughly one-third of the claimants in both the 
treatment group and the control group who received business income in the first year did not receive any business 
income the next year and may have closed. The fact that the business failure rate appears to be similar for 
treatments and controls is consistent with the results of the hazard rate analyses described earlier in this chapter. 

The much larger fraction of controls with open businesses than those who reported jKJsitive business ir come 
in any of the three years is likely due to individuals who are not active in business not bothering to officially close 
the business for some time, but rather keep the business license active by simply reporting "no activity" on their 
sute tax return. Moreover, since we obtained the DOR data in July 1993, information on whether the business was 
still open was based on available data at that time. Because most claimants in our sample were required to report 
information to DOR only at the end of a year, and were under no obligation to report a business closure until the 
end of the year (or later) ~ six months after we received the data -- it is likely that these data overstate the number 
of open businesses. 
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Impacts on Gross Business Income. The Washington State Department of Revenue also 
maintains records of reported gross business income. This includes gross sales as well as 
income received for services provided, which although somewhat different than self-employment 
earnings, is a valuable indicator of the level of activity of businesses established. In Table 7.21, 
we provide information on gross business income in each of the three years for all SEED 
participants, regardless of whether the person established a business; thus, individuals who did 
not start a business during the period (and who are not included in the DOR records) are 
assigned zero values for their gross business income. 

As shown in the first column of Table 7.21, we see that the entire control group averaged 
$1,935 in gross business income in 1990, $4,734 in 1991, and $3,637 in 1992.*^ A 
comparison of these amounts with the earnings from self-employment reported in the followup 
surveys in Table 7.4 indicates that gross business income based on administrative records is 
much greater. This difference is not evidence that the survey income information were 
underreported, but rather represents a fundamental difference in what is being measured. 
Specifically, business income reported in the DOR records include gross sales, as well as income 
from services provided, whereas the survey attempts to measure earnings from self-employment, 
which include deductions for business expenses. 

In the second column of Table 7.21, we show that the average gross business income 
among all treatment group members is considerably larger than for control group members, 
growing from about $4,135 in 1990 to over $11,000 in both 1991 and 1992. Thus, the 
estimated impact of SEED on gross business income is substantial and statistically significant 
in each of the three years. In particular, based on the regression-adjusted results, we estimate 
the impact of SEED on gross business income to be $2,208 in 1990, $6,836 in 1991, and nearly 
$8,000 in 1992." Given the mean values of the control group, these impact estimates 



** If one instead averaged only over those individuals who had positive business income in the year, the average 
gross income of control group members active in business is about $15,000 in 1990, $32,00 in 1991 "uid $28,000 
in 1992. 

" It is interesting to note that among those individuals in the treatment group who had positive business income 
in the year, the average gross income of treatment group members active in business is about $15,000 in 1990, 
$48,000 in 1991 , and $59,000 in 1992. A comparison of these figures with the control group mdicates that although 
the average gross income among those with an active business is similar iji 1990, the treatment group members with 
an active business receive much higher business income in 1991 and 1992 than the controls with an active business 
in those years. 
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correspond to increases in gross business income in excess of I'X) percent in each of the three 
years. 



Table 7.21 
Gross Business Income 



Year 


Control 


Treatment 


Difference 


Impact 


1990 


$1,935 


$4,135 


$2,200" 


$2,208"' 


1991 


$4,734 


$11,172 


$6,438'" 


$6,836"' 


1992 


$3,637 


$11,322 


$7,685"' 


$7,977"' 



Impacts on Business Tax Payments. An important benefit of small business startups 
from the State's perspective is the potential tax revenue generated. In Washington State, this 
includes both sales taxes paid by businesses on products sold, as well as business and occupation 
(B&O) taxes. In Table 7.22 we provide information on taxes paid by businesses operated by 
treatment and control group members for the 1990-1992 period. 

As this table indicates, over aU control group members, the average amount of sales taxes 
paid in 1990 was $46 and this increased to $121 in 1991 and declined to $100 in 1992. Business 
and occupation taxes paid by control group members followed the same pattern, but were much 
lower in each year ($12, $32 and $30, respectively). As shown in the second column, the mean 
values of taxes paid among all treatment group members is somewhat higher than controls in 
each of the three years. Moreover, although the differences are not statistically significant for 
1990, during which the average level of business activity was relatively low, the differences are 
statistically significant in both 1991 and 1992. Specifically, the regression-adjusted estimates 
indicate that SEED increased sales taxes paid by $100 per treatment group member in 1991 and 
by $214 in 1992. Similarly, the results indicate that SEED increased the business and 
occupation taxes paid by about $56 per treatment group member in both 1991 and 1992. 
Consistent with the results described earlier, these impacts on tax payments correspond to 
roughly 100 percent of the control group mean or larger. Taken together, the results indicate 
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an increase in total State tax receipts per treatment group member due to SEED of over $150 
in 1991 and about $270 in 1992. 

Table 7.22 
Business Tax Payments 





v-uui'rui 


J. n^iiucuv 






Sales Taxes 
1990 


$46 


$81 


$35 


$37 


1991 


$121 


$212 


$91" 


$100" 


1992 


$100 


$304 


$204"' 


$214"' 


B&O Taxes 
1990 


$12 


$25 


$13 


$13 


1991 


$32 


$85 


$53"' 


$56"' 


1992 


$30 


$84 


$54"* 


$56"' 



SEED Impacts on Earnings, Employment and Employees 

As described in Chapter 6, in addition to obtaining survey data on wage and salary 
employment in Washington State, we also obtained UI Wage Records for treatment and control 
group members who worked in covered employment in the post-program period. In particular, 
we obtained information on wages and hours worked in covered employment for the quarter the 
claim was filed and for the following nine calendar quarters. These data were used to construct 
measures of whether claimants worked in covered employment in the first and second years 
following enrollment in SEED, hours worked and earnings during the first two years, as well 
as the composition of total earnings by quarter.** 

UI Wage Records provide a valuable alternative source of information concerning the 
earnings of SEED participants. However, because these records are state-based, it is impossible 
to distinguish individuals who work across the border in a different state ~ either in self- 
employment or in wage and salary employment- from individuals who do not work at aU. This 



•* By the first (second) year following SEED enrollment, we mean the first (second) four complete calendar 
quarters following the quarter in which the claimant filed for UI benefits and entered the demonstration. 
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is particularly problematic for claimants in the Vancouver site -- who comprise roughly one- 
quarter of the SEED analysis sample ~ most of whom live within a few minutes of the Oregon 
border and the Portland metropolitan area. Analysis described in the interim report indicated 
that the impacts on earnings measures based on UI Wage Records are sensitive to whether 
Vancouver claimants are included in the analysis. Specifically, we found that the impacts on 
employment and earnings outcomes are biased towards zero when Vancouver claimants are 
included. Consequently, the results presented below exclude Vancouver claimants for the 
employment and earnings analysis based on UI wage records. It should be emphasized, 
however, that the pattern of the findings and the broad conclusions are not affected by the 
exclusion of Vancouver claimants from this analysis. 

The rest of this section is organized as follows. First, we examine SEED impacts on the 
likelihood of employment and hours worked in each of the first two years following filing a 
claim and entering the demonstration. We then examine program impacts on earnings in 
covered employment using both annual and quarterly measures. The section concludes with a 
description of the impacts of SEED on job creation (i.e., number of employees) and wages paid 
to employees based on UI Wage Records. 

Impacts on Likelihood of Working \n Covered Employment. We begin by examining 
the impacts of SEED on the likelihood of working in (covered) wage and salary employment 
during each of the first two years after filing a UI claim and entering the demonstration. This 
outcome measure is defined to be 1 if UI Wage Records indicated the claimant had positive 
earnings in any of the first (second) four quarters after filing his/her claim and zero otherwise. 
In Table 7.23, we report the impact results for these measures of the likelihood of working in 
(covered) wage and salary employment. 

The results in Table 7.23 indicate that 82.6 percent of controls worked in covered 
employment at some point during the first year after filing the UI claim and entering the 
demonstration. Among all treatment group members, 78.5 percent worked in covered 
employment during the same period. The unadjusted and regression-adjusted impacts are similar 
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and indicate that SEED significantly (at the .05 level) reduced the likelihood of working in 
covered wage and salary employment by about 4-5 percentage points in Year 1 

The results for the second year follow the same pattern, although the rate of employment 
for both groups is somewhat lower and the statistical significance of the difference between 
treatments and controls is slightly weaker. Specifically, we find that 76.6 percent of controls 
worked in covered wage and salary employment in Year 2, as compared to 72.7 percent of 
treatment group members, leading to an unadjusted difference of -3.9 percentage points and a 
regression-adjusted impact of -4.3 percentage points which is significant at the . 10 level. Thus, 
consistent with the results based on followup survey data presented earlier in this chapter, it 
appears that SEED has a negative and statistically significant impact on the likelihood of working 
in (covered) wage and salary employment, and that this difference persists through the first two 
years after entering the demonstration. 



Table 7.23 
Employment Status 



Year 


Control 


Treatment 


Difference 


Impact 


Worked Year 1 
Worked Year 2 


82.6% 
76.6% 


78.5% 
72.7% 


-4.1%' 
-3.9% 


-4.7%" 
-4.3%' 



Impacts on Hours Worked in Covered Employment. To understand the intensity of 
employment in wage and salary jobs it is important to examine the amount of time worked in 
such jobs. Washington State is the only State in the U.S. that contains information on hours 



^ Although the SEED demonstration was designed to attract individuals interested in self-employment who were 
not job-attached, it is likely that some of the claimants who worked in covered wage and salary employment in Year 
1 had returned to their same previous employer. To examine this issue, we calculated the percentage of individuals 
who returned to work to their major employer ~ the one they earned the most wages from in the year before filing 
for UI benefits ~ at some time during Year 1. Although such a measure does not capture whether the individual 
directly returned to work at the previous employer after a period of unemployment, it docs provide valuable 
information on whether s/he returned to work with the previous employer at some time within a year. Our resulty 
indicate that 25.0 percent of the control group returned to work for their previous employer during Year 1 and that 
a similar percentage of treatment group members (23.7 percent) also returned to work for their previous employer, 
a difference which is not statistically significant. 
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worked in covered employment in its UI Wage Records system. In Table 7.24, we report the 
impacts of SEED on hours worked in covered employment during the first two years following 
enrollment in the demonstration. 

The findings for hours worked mirror the results for the likelihood of employment and 
indicate that SEED has a negative impact on working in (covered) wage and salary employment 
that persists over the observation period. For example, average hours worked in covered 
employment among aU controls in Year 1 was 875. In contrast, treatment group members 
overall averaged 745 hours in covered wage and salary employment during the first year, 
resulting in an unadjusted reduction of 130 hours. The regression-adjusted estimate of -138 
hours is statistically significant and corresponds to nearly a 16 percentage point reduction in 
hours worked during the first year." 



Table 7.24 
Hours Worked in Covered Employment 



Year 


Control 


Treatment 


Difference 


Impact 1 


Year 1 


875 


745 


-130'" 


-138'" 


Year 2 


986 


895 


-91* 


-97" 1 



The results for the second post-enrollment year follow the same general pattern, although 
the level of hours worked increase somewhat. Specifically, during Year 2, controls average 986 
hours worked in covered employment, as compared to 895 for treatment group members. This 
unadjusted difference of -91 hours is significant at the .10 level and the regression-adjusted 
difference of -97 hours is significant at the .05 level, indicating that the SEED program reduced 
hours worked in covered employment, and that this effect persisted over the two-year 



** ITie difference in hours worked in the first year is in large part due to a difference in work mtensity among 
workers, and not due to the difference in the likelihood of working. Fov example, among the 82.6 percent of 
controls who worked at some point during Year 1, average hours worked were 1,060. This compares to 950 hours 
worked among the 78.5 percent of working treatment group members, and corresponds to a 110 hour reduction m 
hours worked, or roughly a 10 percent reduction evaluated at the control group mean. 
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observation period/' This is consistent with the impacts reported above for the number of 
months in wage and salary employment based on the followup survey data. 

Impacts on Earnings in Covered Employment. These differences in the likelihood of 
working in covered employment, coupled with overall reductions in hours worked, suggest 
potentially important differences between treatments and controls in earnings from (covered) 
wage and salary employment. In Table 7.25, we report the summary impact results based on 
UI Wage Records for the first two years after entering the demonstration, as well as detailed 
results by post-enroUment quarter. As shown in the first row of this table, during the first 
complete year after filing the UI claim and entering the demonstration, controls earned $10,029 
in (covered) wage and salary employment, as compared to only $8,675 for treatment group 
members.™ Both the unadjusted reduction of $1,354, and the regression-adjusted impact 
estimate of -$1,584 are statistically significant and indicate that, consistent with the followup 
survey results presented earlier, SEED reduced the wage and salary earnings of the treatment 
group. In fact, the impact estimate for annualized earnings of -$1,477 for Wave I is quite 
similar to our estimate based on UI Wage Records. 

The results for the second complete year after filing the UI claim follow the same general 
patterns, although the average levels of earnings of both groups are slightly higher and the 
precision of the estimated program effect is somewhat less. Specifically, during Year 2, controls 
averaged $12,336 in (covered) wage and salary earnings, as compared to $1 1,328 for treatment 
group members. The unadjusted difference estimate of -$1,008 is not statistically significantly 
different from zero, although the regression-adjusted estimate of -$1,252 is statistically 
significant at the .10. level. Based on these results, it appears that the reduction in wage and 
salary earnings due to SEED persists through at least the first two years after enrollment in the 
program, although there is no evidence that the effect becomes larger over time. 



" In contrast to Year 1, the differences in hours worked among workers are reasonably small in Year 2. 
Specifically, among claimants who worked in some covered wage and salary employment in Year 2, the average 
hours worked was 1,287 for controls and 1,231 for treatment group members. 

™ Given that the average earnings of control group members in covered employment in the year before filing 
for UI benefits was about $21 ,000, control group members earned less than one-half of their pre-UI annual earnings 
in the first subsequent year. 
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In the remainder of Table 7.25 we provide additional evidence on the time pattern of 
program impacts on wage and salary earnings by decomposing the annual results into separate 
quarters. As these results indicate, the average earnings in covered employment of control 
group members increase consistently over the six complete quarters following SEED enrollment, 
from $1,685 in the first quarter to $3,184 in the sixth quarter, and then remain roughly at the 
level (or slightly below) over the next three quarters. This pattern is generally expected as 
control group members return to work in covered employment over time. As shown in Table 
7.22, treatment group members follow the same general pattern, although their average earnings 
are consistently below that of controls in every quarter. Moreover, the regression-adjusted 
impact estimates indicate that SEED significantiy reduced the quarterly earnings of ti«atinent 
group members by from about $320-470 in each of the four quarters during the first year. The 
quarterly earnings impacts appear to be somewhat smaller in subsequent quarters (about $250- 
380) and are estimated with less precision, with only one of the four quarterly impact estimates 
being statistically significantiy different from zero at conventional levels. 

Table 7.25 
Earnings in Covered Employment 



Year/Quarter 


Control 


Treatment 


Difference 


Impact 


Year 1 


$10,029 


$8,675 


-$1,354" 


-$1,584*** 


Year 2 


$12,336 


$11,328 


-$1,008 


-$1,252* 


Quarter 1 


$1,685 


$1,355 


-$330" 


-$378" 


Quarter 2 


$2,436 


$2,053 


-$383" 


-$468"' 


Quarter 3 


$2,827 


$2,556 


-$271' 


-$323' 


Quarter 4 


$3,081 


$2,710 


-$371" 


-$415" 


Quarter 5 


$3,150 


$2,915 


-$235 


-$300 


Quarter 6 


$3,184 


$2,869 


-$315 


-$383' 


Quarter 7 


$3,064 


$2,881 


-$183 


-$248 


Quarter 8 


$2,938 


$2,663 


-$275 


-$322 


Quarter 9 


$3,134 


$2,922 


-$212 


-$266 
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Impacts on Business Employees Using UI Wage Records. Because businesses must 
submit to the Employment Security Department (ESD) information concerning quarterly wages 
paid to each employee, it is also possible to utilize UI Wage Records to examine the impacts of 
SEED on the number of employees hired by demonstration participants. In this way, we could 
provide an alternative assessment of job creation impacts, which do not rely on interview self- 
reports. 

To address this issue, we first used data to identify businesses owned or operated by 
SEED treatment or control group members during the post-SEED period. We then obtained 
from ESD all quarterly wage records available at that time for all individuals who worked for 
the businesses of any treatment or control group members. At the time the information was 
obtained, twelve quarters of data from the third quarter of 1990 through the second quarter of 
1993 were available.^' By linking individual worker records for each quarter to a specific 
business, we could determine the number of different employees in a quarter a business had, as 
well as total wages paid. For SEED treatment and control group members who were not self- 
employed, as well as those who were self-employed but who did not submit any records of 
wages paid, we set the number of employees and wages paid to employees to be zero. Below, 
we first examine impacts on number of employees. 

As shown in Table 7.26, control group members consistently submitted quarterly wage 
reports for a small number of workers. Although this ranged from .03 employees on average 
for the third quarter of 1990 to .06 employees during the second quarter of 1992, controls hired 
.04 employees on average in 8 of the 12 quarters shown in Table 7.26. In contrast, the mean 
number of employees of the treatment group was .06 during the third quarter of 1990, and then 
increased reasonably steadily to .22 by the end of this three-year period. 

The unadjusted difference estimates and the regression-adjusted estimates are statistically 
significant in a)l of the 12 quarters except the first one, and indicate that SEED had a positive 
impact on job creation. Specifically, over the latest several quarters, we estimate that SEED 
increased the number of employees by about .17 per treatment group member on average. It 



" Because data were not readily available for the entire post-program period, it was not possible to create 
outcome measures that corresponded to the first nine quarters following the quarter of enrollment into SEED. As 
a result, we decided to analyze the data separately for each calendar quarters. 
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is important to note that this estimate is not dissimilar to the .25 estimated number of employees 
per treatment group member based on the Wave n followup survey described earlier. Over the 
entire sample of 1,507 treatment and control groi^ members, these results suggest that controls 
as a group employ approximately 30 workers each quarter over all businesses, as compared to 
about 165 employees woridng for treatment group members, a net increase of 135 workers. 



Table 7.26 
Number of Employees 



Year/Quarter j 


Control 


Treatment 


Diiterence 


Impact 


1990, Q3 


.03 


.08 


.05 


.05 


1990, Q4 


.04 


.12 


.08" 


.09" 


1991, Ql 


.04 


.10 


.06* 


.06* 


1991, Q2 


.06 


.16 


.10" 


.11" 


1991, Q3 


.03 


.19 


.16"- 


.16"' 


1991, Q4 


.04 


.17 


.13"' 


.14"- 


1992, Ql 


.04 


.17 


.13"' 


.12"- 


1992, Q2 


.05 


.20 


.15"' 


.15"' 


1992, Q3 


.04 


.21 


.IT" 


.17"' 


1992, Q4 


.04 


.20 


.16"- 


.16"' 


1993, Ql 


.04 


.21 


.17"' 


.16"' 


1993, Q2 


.04 


.22 


.18"' 


.IT" 



Impacts on Wages Paid to Employees Using UI Wage Records. We also used UI 
Wage Records to examine the impacts of SEED on the total wages paid to the employees of 
businesses operated by treatment and control group members. The results by calendar quarter 
are shown in Table 7.27. Consistent with the results for number of employees, for the most 
part, these results indicate a generally increasing pattern of total wages paid to employees for 
controls and a much stronger increasing pattern for treatment group members. For example, 
in the third quarter of 1990, at the time when the last claimants were bemg enrolled in SEED, 
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control businesses on average paid $12 in employee wages, as compared to $82 for treatment 
group businesses. The unadjusted difference of $70 is statistically significant at the .05 level 
as is the regression-adjusted estimate of $73. 

By two to three years later, the employee wage bill for control businesses had increased 
on average to about $75-100 per quarter, as compared to $475-500 per quarter for treatment 
group businesses. This resulted in difference estimates and regression-adjusted impact estimates 
of approximately $375-400 per quarter increase in the total wages paid to employees due to 
SEED. Over the entire three-year observation period of these data, these results indicate that 
SEED increased total wages paid to employees by about $2.5 million. 



Table 7.27 
Total Employee Wages Paid 



Year/Quarter 


Control 


Treatment | 


Difference 


Impact 


1990, Q3 


$12 


$82 


$70" 


$73" 


1990, Q4 


$22 


$194 


172"' 


$182"' 


1991, Ql 


$43 


$195 


$152" 


$159" 


1991, Q2 


$89 


$274 


$185" 


$195" 


1991, Q3 


$54 


$328 


$274"' 


$284"' 


1991, Q4 


$65 


$357 


$292"' 


$293"' 


1992, Ql 


$56 


$379 


$323"' 


$324"' 


1992, Q2 


$70 


$426 


$356"' 


$352"' 


1992, Q3 


$73 


$475 


$402"' 


$402"' 


1992, Q4 


$85 


$492 


$407"' 


$402"' 


1993, Ql 


$108 


$509 


$401"' 


$397"' 


1993, Q2 


$104 


$486 


$382""' 


$370"' 
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Impacts of SEED for Various Claimant Subgroups 



We also examined whether the impacts of SEED on the self-employment and wage and 
salary employment outcome measures varied by numerous claimant characteristics, including 
gender, race, site, and whether the individual intended to return to the previous employer at the 
time of application to SEED. To test for differential program impacts across subgroups, we 
added interaction terms (e.g., treatment status interacted with subgroup characteristic(s)) to the 
basic multivariate regression model described earlier in this chapter. In this section we describe 
the general patterns of the subgroup results for the main outcome measures derived from the 
followup survey data that were statistically significant in both Wave I and Wave H; readers 
interested in additional details can refer to Appendix C. 

First, we describe the patterns of subgroup impact differences for the self-employment 
outcome measures. The results of these analyses indicate that the SEED Demonstration had a 
significantly greater impact on females than on males in increasing the likelihood of self- 
employment.''^ Specifically, the regression-adjusted impact estimate indicates that the program 
had a 16-17 percentage point greater impact in increasing the likelihood of self-employment for 
women than men. However, there were no significant differences in the impact of SEED by 
gender on the number of months in self-employment or on the likelihood of being self-employed 
at either followup survey.'^ This suggests that the rate of self-employment terminations are 
higher among women than men. 



" It is not clear why the program had a greater impact on women's participation in self-employment over the 
observation period than on men's participation. One possible explanation is that women are more receptive to the 
types of services provided by the SEED program. An alternative explanation is that the program may have provided 
superior assistance for the types of enterprises that women were most interested in. A third explanation may be 
that women have a more difficult time m obtaining financing and that the SEED lump-sum payment may have been 
relatively more helpful to women in overcoming this disadvantage. 

" The Wave I results indicated that SEED had a larger impact in increasing time in self-employment for females 
than males, but this effect was much reduced and not statistically significant for the entire 33-month observation 
period. 

^* The results of the hazard rate analysis of the time until transition from the first self-employment spell is 
consistent with this speculation. Specifically, among those who started a self-employment spell, the estimated 
hazard of terminating the self-employment spell is higher for male controls than male treatment group members, 
whereas the estimated hazards are consistently higher for female treatment group members than for female controls. 
However, these differences are not significant at conventional levels, in part due to the very small sample sizes 
involved. 
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In contrast to the results for self-employment outcome measures, our results indicate that 
the effects of SEED on wage and salary outcome measures are widespread and not concentrated 
in specific subgroups. Specifically, none of the interaction terms were significant for both Wave 
I and n.^^ 

The final set of subgroup results concern the outcome measures for combined self- 
employment and wage and salary employment. For the likelihood of being employed at some 
time during the observation period, there are two consistent patterns of interaction effects for 
both Wave I and H. In puiticular, the results indicate that the impact of SEED on the likelihood 
of being employed in some Cype of job was greater for whites than for nonwhites. 

There are also strong differences in the impacts of SEED on months worked and on total 
combined earnings by business ownership status at application. For example, using the Wave 
I data, the impact of SEED on annualized time worked for those who owned a business at the 
time of application was 4. 1 months per year, as compared to 1.9 months per year for those who 
previously owned a business (but not at the time of application), and as compared to .6 months 
per year for those who had never been self-employed. Moreover, it seems that the impact of 
SEED on total earnings is positive and quite large for those who were business owners at 
application, and the impact is relatively small and negative for those who were non-owners at 
application. The results for the entire 33-month observation period follow the identical pattern. 

Summary 

In this chapter we presented our detailed results concerning the impacts of SEED on 
numerous self-employment and wage and salary employment outcome measures, as well as on 
job creation and UI benefits receipt. In this section, we briefly summarize the highlights of 
these fmdings. In particular, we focus on the main self-employment and wage and salary 



" For Wave I, our results indicated that there were differences in the impact of SEED on the likelihood of wage 
and salary employment and on months in wage and salary employment between those in the top quartile of the prior 
earnings distribution than for those in the lowest quartile. In particular, the impact of SEED was to reduce the 
likelihood of wage and salary employment and number of months worked for those in the top quartile by a greater 
amount than those in the bottom quartile. Although the pattern was similar for Wave D, the interaction effect was 
not statistically significant at conventional levels. 
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employment and earnings measures based on the followup surveys and on the receipt of UI 
benefits that contribute to the' benefit-cost results described in Chapter 10. 

The main impact results of the SEED program tested in Washington State are 
summarized in Table 7.28. As these results indicate, the SEED program had a very large and 
positive impact on the self-employment experiences of UI claimants that persisted throughout 
both the Wave I and Wave n survey. Specifically, treatment group members are estimated to 
be much more likely than controls to have a self-employment experience (by about 22-25 
percentage points); to spend more time per year in self-employment (by 2.0-2.3 months); and 
are more likely to be still self-employed at the time of the followup surveys (by 12-16 percentage 
points). Consistent with these results, we fmd that the SEED program increased claimants' self- 
employment earnings by over $1,600 per year. These estimated impacts are quite large and 
correspond to effects that are roughly from 50 percent to over iOO percent of the control group 
mean. 

In contrast to the estimated impacts on self-employment outcomes, the impacts on wage 
and salary employment measures are consistently negative and indicate that SEED significantly 
reduced these indicators of wage and salary employment in both the short- and long-term. 
Specifically, SEED reduced claimants' likeUhood of working in wage and salary employment 
by 6 percentage points, reduced number of months per year working in wage and salary 
employment and reduced earnings from wage and salary employment by from about $1,400- 
$1,800 per year. These effects, while statistically significant and economically important, 
represent smaller percentage changes. Specifically, these impacts correspond roughly to 10-20 
percent effects evaluated at the mean of the control group. 

In the bottom panel of Table 7.28 we present the impacts of SEED on total employment 
and earnings outcomes. These combined results do not follow as strong or as consistent of a 
pattern as the separate results for both self-employment and wage and salary employment 
measures. The first three rows taken together suggest that the SEED program was fairly 
successful in assisting claimants become reemployed. Specifically, we find that the program 
significantly increased the likelihood of claimants' finding employment at some time in Wave 
I, increased the number of months worked per year (by 1.1-1.3 months per year), and slightly 
increased the likelihood of being employed at the time of the Wave n survey. On the other 
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Table 7.28 
Washuigton Summary 
Program Impacts 







WaveH 


Self-Employment 






Percent Self-Employed 
Since Random Assignment 


+25%'" 


+22%'" 


Time Self-Employed 
Since Random Assignment 


+2.3'" 
months 


+2.0"' 
months 


Percent Self-Employed at the End 
of Observation Period 


+ 16%'" 


+ 12%'" 


Earnings from Self-Employment 
Since Random Assignment 


+$1,596" 


-l-$l,675" 


Wage and Salary Employment 






Percent Employed in W&S 
Since Random Assignment 


-7%"* 


-6%'" 


Time Employed in W&S 
Since Random Assignment 


-0.6" 
months 


-0.7" 
months 


Percent Employed in W&S at the 
End of Observation Period 


-7%" 


-3%' 


Earnings from W&S 
Since Random Assignment 


-$1,407* 


-$1,780" 


Total Employment 






Percent Employed 

Since Random Assignment 


+A%'" 


-1-0% 


Time in Employment 
Since Random Assignment 


+ 1.3'" 
months 


+ 1.1'" 
months 


Percent Employed at the End of 
Observation Period 


-1-3% 


+6%" 


Total Earnings 

Since Random Assigmnent 


-f-$566 


+$289 
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hand, the impacts on total earnings are insignificantly different from zero for both Wave I and 
n. Thus, it appears that SEED did not increase claimants' total earnings from all types of 
employment. 

We also examined in detail the impacts of SEED on total benefits receipt. Although not 
reported in this table, our results indicate that after taking into account the lump-sum payment 
provided to treatment group members who met the required milestone? for starting a business, 
SEED increased total benefit payments by $1,098 during the initial benefit year. Because the 
estimated impact of SEED on subsequent receipt of UI was small and negative, our results 
indicate that SEED increased total benefits by $1,013 per claimant over the entire observation 
period. 
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MASSACHUSETTS DEMONSTRATION IMPACTS 



In this chapter we present the impacts of the Massachusetts Enterprise Project. The 
results are presented in two sections. In the first section, we present our main impact results 
based on an evaluation of Cohorts 1 and 2. In the second section of this chapter, we evaluate 
Cohort 3 separately. As discussed in Chapter 4, exogenous developments in Massachusetts 
altered the implementation of the demonstration, especially during the period when Cohort 3 was 
enrolled. As a result, we concluded that it is not appropriate to combine all three cohorts for 
the impact analysis; instead, we combine only Cohorts 1 and 2 for the main impact analysis. 
We present the Cohort 3 impact results largely for completeness. Further analysis is required 
to determine if and how all three cohorts can be pooled to obtain an overall impact estimate. 

As we did in the evaluation of the SEED Demonstration impacts, we measure progxam 
impacts by comparing the experiences of treatment group members with the experiences of 
control group members following random assignment. Within each of the two sections in this 
chapter, we follow the same presentation pattern as was used earlier for the SEED evaluation. 
First, we describe the demonstration impacts on self-employment outcomes, including the 
likelihood of becoming self-employed, total time in self-employment, the likelihood of being 
self-employed at the time of the followup surveys, and total earnings from self-employment. 
We then examine the impact on similar measures for wage and salary employment. Next, we 
combine self-employment with wage and salary employment outcomes and examine the 
demonstration impacts on total employment and earnings. Following this evaluation of 
employment and earnings impacts, we present the demonstration impacts on such outcomes as 
job creation, length of UI spells, and UI benefits receipt. Throughout the chapter, we also 
provide information about the impact of the demonstration on key claimant subgroups. 
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As in Chapter 7, we present results for two overlapping time periods. First, we present 
impacts based on data obtained from the Wave I survey, which, in Massachusetts, was 
administered on average 19 months after random assignment. Second, we present impacts based 
on a combination of Wave I and Wave n survey data. These data are used to create a 31 -month 
(on average) observation period, beginning with random assignment and ending with the Wave 
n survey. In the tables below, impact estimates derived from the 19-month observation period 
are referred to as Wave I results, while impact estimates derived from the 31 -month observation 
period are referred to as Wave II results. 

Impacts Based on Cohorts 1 and 2 

In this section, we present impact estimates based on the combined samples from Cohorts 
1 and 2. Combining these two cohorts yields a total of 263 treatment group members and 258 
control group members. The 86 percent Wave I survey response rate yields a total sample of 
449 for the Wave I impact analysis (229 treatment group members and 220 controls). The 
89 percent Wave n response rate yields a total sample of 398 for the Wave n impact analysis 
(204 treatment group members and 194 controls). 

Self-Employment Impacts 

This subsection covers the impacts of the Enterprise Project on self-employment 
outcomes. The next subsection examines the demonstration impacts on wage and salary 
employment outcomes. In the final subsection, we combine the self-employment with wage and 
salary employment outcomes and evaluate demonstration impacts on combined employment and 
earnings outcomes. 

Impacts on Likelihood of Self-Employment Experience. As in the SEED 
Demonstration, the Enterprise Project provided business training, counseling, and financial 
assistance to targeted UI claimants who were interested enough in self-employment to both attend 
an information session and submit an application, and who were randomly selected into the 
treatment group. To the extent that the demonstration services provided effective assistance to 
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treatment group members, one would expect more treatment group members than control group 
members to enter self-employment. 

The results in Table 8.1 indicate that, indeed, treatment group members were much more 
likely than control group members to enter self-employment. Specifically, within the first 19 
months after random assignment (Wave I), 52 percent of the treatment group, as compared with 
35 percent of the control group, were self-employed at some point (i.e, had at least one self- 
employment spell).'* The 17 percentage point difference in the likelihood of self-employment 
for the two groups is statistically significant (at the .01 level) and corresponds to about 50 
percent of the control group mean. 



Table 8.1 

Self-Employment Experience Since Random Assignment 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact 


Wave I 


35% 


52% 


+ 17%*" 


+17%*** 


Wave 11 


47% 


58% 


+ 11%** 


+ 12%** 



As indicated in Table 8.1, additional control and treatment group members entered self- 
employment between the two surveys. That is, for the control group, the likelihood of having 
a self-employment spell increase^ from 35 percent in Wave I to 47 percent in Wave H. For the 
treatment group, the self-employment likelihood increased from 52 to 58 percent. As a result 
of the greater influx by control group members into self-employment after the Wave I survey, 
the simple difference in the likelihood of having a self-employment experience between the 
treatment and control groups declined from 17 percentage points in Wave I to 11 percentage 



The Wave I results reported here differ slightly from the Wave I findings presented in our earlier report 
(Benus, Wood and Grover, 1994), although all of the main findmgs are qualitatively similar. The differences are 
due to nunor definitional changes in the self-employment experience variable. 
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points in Wave H. The difference in Wave n corresponds to less than 25 percent of the control 
group mean. This difference remained statistically significant (at the .05 level). 

The regression-adjusted impact estimates of the likelihood of having a self-employment 
experience, reported in the last column of Table 8.1, are quite similar to the simple-difference 
estimates reported in the third column." Based on these regression-adjusted impacts of 
17 percentage points in Wave I and 12 percentage points in Wave H, we conclude that the 
Enterprise Project had a large positive impact on the likelihood of becoming self-employed. 

The impact results reported in the last column of Table 8. 1 are based on logit regression 
models that control for a number of covariates. Although not reported in the table, the 
regression results indicate that several of the covariates significantly affect the likelihood of 
entering self-employment in both the Wave I and Wave n.^* 

Impacts on Time in Self-Employment. In this section we evaluate the impact of the 
Enterprise Project on the time spent in self-employment after random assignment. This analysis, 
uses annualized measures of the number of months in self-employment following random 
assignment. Specifically, using information on start dates and end dates of self-employment 
spells from the followup surveys, annualized time in self-employment is calculated as the 
proportion of the period from random assignment to the interview date that an individual sample 
member was self-employed multiplied by 12. 

As shown in Table 8.2 and consistent with the results described above, treatment group 
members spent more time in self-employment on average than controls. On an annualized basis, 
treatment group members spent 2.5 months per year in self-employment during the first 19 
months after random assignment (Wave I) and slightly more (2.6 months per year) during the 
entire 31 -month observation period (Wave H). Treatment group members, on average, spent 
approximately one month more per year in self-employment than control group members. 



The covariates used in all the Massachusetts regressions are presented and defined in Appendix A. 

In Appendix B, we identify all the significant covariate coefficients, as well as their signs, for each of the 
regressions presented. In this regression, for example, we find that in both the Wave I and Wave 11 regression 
models males are more likely to enter self-employment, while those who previously worked in the service industry 
are less likely to enter self-employment. 

132 

Er|c Ibu 



Chapter 8 • Massachusetts Demonstration Impacts 



Similar results are obtained from the regression-adjusted impacts." Based on these results, we 
estimate that treatment group members on average worked approximately 2.2 months more in 
self-employment than controls during the 31-month observation period.*" 

Table 8.2 



Time in Self-Employment Since Random Assignment 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact 


Wave I 


1.4 


2.5 


-f-l.l"' 


-f-1.0'" 




months 


months 


months 


months 


Waven 


1.7 


2.6 


+0.9" 


-f-0.8' 




months 


months 


months 


months 



Impacts on Self-Employment at Tune of the Followup Surveys. The results described 
above indicate that treatment group members were more likely than control group members to 
have a self-employment spell during the observation period, and that they spent more time self- 
employed than controls. Although these results confirm that the Enterprise Project was 
successful in encouraging business startups, they do not necessarily indicate program success. 
A better indicator of program success is the proportion of individuals remaining in self- 
employment at the time of the followup surveys. 

In Table 8.3, we present information on the proportion of treatment and control group 
members who were self-employed at the time of each followup survey. As this table indicates, 
40 percent of the treatment group was self-employed at the tune of the Wave I survey, as 
compared to 29 percent of the control group. This corresponds to a 11 percentage point 
difference, which is about a 38 percent effect at the control group mean (statistically significant 
at the .05 level). By the time of the Wave n survey, the number of controls in self-employment 



ERIC 



^' Only one covariate was statistically significant in both the V/ave I and Wave 11 regression models: being in 
the service industry. Those who previously worked in the service industry spent less time in self-employment after 
random assignment than those who worked in other industries. 

*• This estimate is derived by multiplying the Wave 11 impact estimate (0.8 months) by 31/12. 
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had increased by 9 percentage points to 38 percent, while the proportion of treatment group 
members in self-employment had only increased sUghtiy (by 3 percentage points) to 43 percent. 
At the time of the Wave EL survey, therefore, the simple-difference estimator declined to 
5 percentage points, which corresponds to only a 13 percent effect of the control group mean 
and is statistically iasignificant. As indicated in the table, the regression-adjusted logit impacts 
(presented in the last column) are similar to the simple-difference estimates (column 3)." 



Table 8.3 

Self-Employment at Two Different Observation Times 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact 


Wave I 


29% 


40% 


+ 11%" 


+ 11%** 


WaveH 


38% 


43% 


+5% 


+5% 



Impacts on Self-Einployment Earnings. In Table 8.4, we present estimates of program 
impact on self-employment earnings. Similar to the measure of time in self-employment, 
earnings from self-employment is constructed as an annualized measure. Moreover, the earnings 
figures have been deflated to 1990 dollars using a CPI index. As shown in Table 8.4, during 
the Wave I observation period, annualized self-employment earnings for the control group 
averaged $1,404 in 1990 dollars.'^ In contrast, annualized self-employment earnings during 
Wave I for the treatment group averaged $2,916. Neither the unadjusted impact of $1,512, nor 
the regression-adjusted impact of $1,218, are significant at the .10 level.'' 



There were no covariates that were stotistically significant in both the Wave I and II regression models for 
the likelihood of being 8clf-«mployed at the time of the followup surveys. 

This average earnings figure includes cases where self-«niployment earnings were reported to be zero. To 
the extent that some of the zero responses are likely to be misrcportcd, the above average earnings figure is likely 
to understate actual sclf-«mployment earnings. 

" The only covariate that significantly affected self-employment earnings in both the Wave I and Wave U 
models was prior employment in the service industry (-). 
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The findings for Wave n are quite similar to the Wave I results. The unadjusted 
annualized impact estimate is $1,188 and the regression-adjusted impact estimate is $1,219. 
Neither of these impact estimates is significant. Thus, we conclude that the Enterprise Project 
did not significantly affect self-employment earnings. 



Table 8.4 



Earnings From Self-Employment Since Random Assignment 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


bapact 


Wave I 


$1,404 


$2,916 


+$1,512 


+$1,218 


Wave n 


$1,439 


$2,627 


+$1,188 


+$1,219 



In summary, the impacts of the Massachusetts demonstration on self-employment 
outcomes are inconsistent. The Enterprise Project had significant positive impacts on some of 
the self-employment outcomes analyzed, while on others the effect was insignificant. For 
example, in both observation periods, treatment group members were more likely than controls 
to have a self-employment experience and to spend more time per year in self-employment. In 
contrast to the above significant impacts, the demonstration did not have a statistically significant 
impact on self-employment earnings during either of the two observation periods. 



Wage and Salary Employment Impacts 

In this section we continue our focus on the employment and earnings impacts of the 
Massachusetts demonstration, but shift our attention to examine the unpacts of the demonstration 
on the wage and salary employment experiences of program participants. 

Impacts on Likelihood of Wage and Salary Employment Experience. In Table 8.5 
we examme the impacts of the Enterprise Project on the likelihood of having a wage and salary 
employment experience during the observation period. These results indicate that approximately 
the same proportion of the treatment group as the control group had at least one wage and salary 
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job during the observation period. During the Wave I observation period, 64 percent of the 
control group and 65 percent of treatment group had at least one wage and salary job; during 
the Wave n observation period, 75 percent of control group members and 71 percent of 
treatment group members had at least one wage and salary job. Neither the Wave I nor the 
Wave n treatment/control group difference is statistically significant, indicating that the 
Enterprise Project did not have an impact on the likelihood of wage and salary employment. 



Table 8.5 

Wage and Salary Employment Experience Since Random Assignment 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact 


Wave I 


64% . 


65% 


+ 1% 


+ 1% 


Waven 


75% 


71% 


-4% 


-4% 



In the last column of Table 8.5 we present the impact results derived from logit 
regression models for the probability of having a wage and salary job during the observation 
period. These results confirm our conclusion that the Enterprise Project did not affect the 
likelihood of having a wage and salary job during either the Wave I or the Wave n observation 
period." 

Impacts on Time in Wage and Salary Jobs. In Table 8.6, we report the impacts of 
the Enterprise Project on annualized months in wage and salary jobs during the two observation 
periods. During the first 19 months after random assignment (Wave I), treatment group 
members worked 3.8 months per year in wage and salary jobs compared to 3.2 months per year 
for controls. The increase of 0.6 months per year was statistically insignificant at the . 10 level. 



** As reported in Appendix B, holding other variables constant, three factors have a statistically significant 
impact on the likelihood of having a wage and salary job in both observation periods. These factors are: prior 
employment in the service industry (+), being a participant in the Milford site (+), and having « benefit start date 
in the first quarter of 1991 (+). 
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The results for the entire 31 -month observation period (Wave II) indicate that both the 
treatment and control groups increased their annual time spent in wage and salary jobs. The 
difference between the two groups in annual time spent in wage and salary employment, 
however, remained small and insignificant (0.3 months). 



Table 8.6 

Time in Wage and Salary Employment Since Random Assignment 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact 


Wave I 


3.2 


3.8 


+0.6 


+0.8" 




months 


months 


months 


months 


Wave n 


4.1 


4.4 


+0.3 


+0.6 




months 


months 


months 


months 



The results for the entire 31 -month observation period (Wave II) indicate that both the 
treatment and control groups increased their annual time spent in wage and salary jobs. The 
difference between the two groups in annual time spent in wage and salary employment, 
however, remained small and insignificant (0.3 months). 

The regression-adjusted impacts shown in the last column of Table 8.6 are somewhat 
higher than the simple-difference estimates. Moreover, the Wave I regression-adjusted impact 
estimate is statistically significant at the .05 level. Thus, during Wave I, the Enteiprise Project 
increased the time spent in wage and salary jobs.*^ 

Impacts on Wage and Salary Employment at Time of the Followup Surveys. In 
Table 8.7, we provide information on the proportion of sample members who were employed 



" Covariates that were statistically significant in both the Wave I and Wave II regression models for time in 
wage and salary en.ployment include whether the claimant was male (-), whether the claimant was previously 
employed in the service industry (+), and whether Ul benefits started during the first quarter of 1991 (+). 
Specifically, the results indicate that holding other factors constant, males spent less time in wage and salary 
employment than females, that the time spent in wage and salary employment was higher for those who were 
previously employed in the service industry, and that claimants whose benefits started in the first quarter of 1991 
spent more time in wage and salary employment than those whose benefits started in other periods. 
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in wage and salary jobs at the time of the two followup surveys. As this table indicates, at the 
Wave I followup survey, 46 percent of control group members and 52 percent of treatment 
group members were employed in wage and salary jobs. This difference of 6 percentage points 
matches the regression-adjusted logit estimate of 6 percentage points. This Wave I estimate is 
statistically significant (at the .10 level). 



Table 8.7 

Wage and Salary Employment at Two Different Observation Times 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


DifTerence 


Impact 


Wave I 


46% 


52% 


+6% 


+6%' 


Wave n 


54% 


56% 


+2% 


+ 1% 



As indicated in the table, the difference between treatment and control groups in the 
likelihood of wage and salary employment is lower at the time of the Wave n survey. 
Specifically, at the time of the Wave n survey, 56 percent of treatment group members were 
employed in wage and salary jobs, as compared to 54 percent of controls. The small 2 
percentage point difference is not statistically significant; the regression-adjusted impact is also 
insignificant. 

Impacts on Wage and Salary Earnings. The results for annualized earnings in wage 
and salary employment are shown in Table 8.8. Consistent with our measure of self- 
employment earnings, earnings in wage and salary employment are expressed in annualized 
terms and measured in constant 1990 dollars. These results indicate that during the initial 19- 
month observation period, treatment group members earned $9,311 per year on average as 
compared to $6,613 per year for control group members.** This yields an unadjusted impact 
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** This average anniulized earnings figure includes some cases where wage and salary earnings were reported 
to be zero. To the extent that some of the zero responses are likely to be misreported, the above average earnings 
figure is likely to understate actual wage and salary earnings. However, this data problem is much less serious than 
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of +$2,698, which is significant at the .05 level. After controlling for other factors, the 
regression-adjusted impact is estimated to be +$3,230, also significant at the .05 level. 



Table 8.8 

Earnings From Wage and Salary Employment Since Random Assignment 

Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact 


Wave I 


$6,613 


$9,311 


+$2,698" 


+$3,230" 


Wave n 


$7,797 


$10,119 


+$2,322* 


+$3,053" 



The results for the entire 3 1 -month observation period are similar. As described in Table 
8.8, average annualized earnings in wage and salary employment for the treatment group over 
the entire observation period was $10,119, as compared to $7,797 for the control group. This 
unadjusted difference of +$2,322 is statistically significant at the .10 level. Moreover, after 
controlling for other factors, the regression-adjusted annualized impact on wage and salary 
earnings is +$3,053, which is statistically significant at the .05 level. Thus, we conclude that 
the Enterprise Project had a positive and significant impact on the annual earnings of participants 
from wage and salary jobs during the observation period." 

Total Employment and Earnings Impacts 

Th results described above indicate that the Enteiprise Project had positive and 
statistically significant impacts on the probability and duration of self-employment, but not on 
self-employment earnings. The results also indicate positive and significant impacts on wage 
and salary employment outcomes (except on the likelihood of wage and salary employment). 



for telf-employment earnings. 

" Hie only covtriatcs that were statistically significant m predicting wage and salary earnings for both Wave 
I and Wave n were whether the claimant was previously employed in the service mdustry (H ) and whether UI 
benefiU started m the first two quarters of 1991 (+). 
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In this section, we present our estimates of the impact of the Enterprise Project on combined 
self-employment and wage and salary employment outcomes. 

Impacts on Likelihood of Employment. In the analysis presented above, we found that 
the Enterprise Project had a positive and significant impact, during both observation periods, on 
the likelihood of having at least one self-employment experience. Moreover, we found that the 
impact of the demonstration on the likelihood of having at least one wage and salary job during 
both observation periods was insignificant. In Table 8.9, we combine the two types of 
employment (self-employment and wage and salary employment) and examine the impacts of the 
demonstration on the likelihood of having at least one employment experience ~ either self- 
employment or a wage and salary job. These results indicate that during the initial 19-month 
observation period, the Enterprise Project had a significant and positive impact on the likelihood 
of employment. Specifically, 93 percent of the ti^atment group was employed in either wage 
and salary or self-employment during this period, as compared to 82 percent of controls. The 
unadjusted 11 percentage point difference is statistically significant at the .01 level. The 
regression-adjusted impact estimate of 11 percentage points is also significant at the .01 level. 



Table 8.9 

Combined Employment Experience Since Random Assignment 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact 


Wave I 


82% 


93% 


-f-11%"' 


-f-11%"* 


Waven 


92% 


97% 


-f-5%" 


-f-5%" 



Based on these results, it is clear that the Enterprise Project significanUy facilitated the 
reemployment of program participants. 

Over the entire 31 -month observation period (Wave H), this positive impact persisted. 
Specifically, during the Wave n observation period, 97 percent of the treatment group and 92 
percent of the control group held a job or were self-employed at some point. The 5 percentage 
point impact, while smaller than the 9 percentage point impact in Wave I, remains statistically 
significant (at the .05 level). Thus, the Enterprise Project not only facilitated reemployment of 
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program participants, but also positively affected the likelihood of employment well into the 
future. 

Impacts on Time in Either Wage and Salary Employment or Self-Employment. The 
results described earlier indicated that the Enterprise Project significantly increased the amount 
of time spent in self-employment during both observation periods. The results also showed that, 
during Wave I, the Enterprise Project also increased the time in wage and salary jobs. To 
examine the overall effects of the Enteiprise Project on time in either self-employment or wage 
and salary employment, we created an annualized measure of combined months employed in 
either type of employment. The total measure is not simply the sum of the separate annualized 
measures of time in self-employment and time in wage and salary employment." Rather, the 
combined measure takes into account information on the dates of each employment experience 
to avoid double-counting periods during which individuals are working at both self-employment 
and wage and salary employment. 

Annualized total time in either self-employment or wage and salary employment is 
presented in Table 8.10. For the entire control group, the average annual time in either type 
of employment during the initial 19-month observation period was 4.5 months. This compares 
to 6.5 months per year for the treatment group. The unadjusted impact of 2.0 months is 
statistically significant at the .01 level, as is the regression-adjusted impact estimate of 2.1 
months. 



Table 8.10 

Total Time in Employment Since Random Assignment 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact j 


Wave I 


4.5 


6.5 


+2.0*" 


+2.1"* 1 




months 


months 


months 


months 


WaveH 


5.8 


7.4 


+ 1.6*" 


+ 1.9"* 




months 


months 


months 


months | 



" As a iesult, the overall impacts of the Enterprise Project on this measure do not equal the simple sum of the 
impacts on time in self-employment and time in wage and salary employment. Moreover, the combined results are 
not equal to the sum of the separate impacts because of differences in sample size due to missing data on one or 
the other components. 
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The results are somewhat higher for the entire 31-month observation period. The 
annualized time ui employment increased for both the treatment and control groups — to 7.4 
months and 5.8 months, respectively - resulting in a simple-difference estimate of 1.6 months 
per year. The corresponding regression-adjusted estimate was 1.9 months per year, which, like 
the simple-difference estimator, was also statistically significant at the .01 level. Thus, we 
conclude that the Enterprise Project unambiguously increased total time in employment by 
approximately two months per year, or nearly 6 months over the entire observation period. 

Impacts on Employment at Time of the FoUowup Surveys. The results of the previous 
sections indicate that the Enterprise Project significantly increased the likelihood of being self- 
employed at the time of the Wave I survey, but not at the time of the Wave n survey. 
Similarly, previous results indicated that employment in wage and salary jobs was also positively 
affected by the demonstration at the time of the Wave I survey, but not at the time of the wave 
n survey. In Table 8. 1 1 , we examine the impact of the Enterprise Project on the likelihood of 
being employed in either type of employment at the time of the two surveys. 



Table 8.11 

Employment at Two Different Observation Times 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact 


Wave I 


68% 


81% 


-l-13%*" 


-l-13%*" 


Waven 


81% 


87% 


4-6% 


-l-6%' 



As indicated in Table 8.11, 68 percent of the control group and 81 percent of the 
treatment group were employed in either self-employment or a wage and salary job at the time 
of the Wave I survey. The simple difference of 13 percentage pomts as well as the regression- 
adjusted impact of 13 percentage points, are both statistically significant at the .01 level. 

At the time of the Wave n survey, 87 percent of treatment group members and 81 
percent of controls were employed in either self-employment or wage and salary employment. 
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The simple dilference of 6 percentage points was not statistically significant, but the 6 percent 
regression-adjusted impact was statistically significant at the .10 level. 

Impacts on Total Earnings. The final results in this section focus on total earnings 
from both wage and salary jobs and self-employment. As shown in I'Ms 8.12, average 
annualized earnings from both wage and salary employment and self-employment over the 19- 
month observation period for the control group was $8,483. Total annualized earnings over the 
same period for the treatment group averaged $13,151. The unadjusted annualized impact of 
$4,668 and the regression-adjusted impact of $4,764 are both significant at the .05 level. 

Table 8.12 



Combined Earnings From Employment Since Random Assignment 
Massachusetts, Cohorts 1 and 2 





Control 


Treatnient 


Dii^erence 


Impact 


Wave I 


$8,483 


$13;151 


+$4,668" 


+$4,764" 


Waven 


$10,056 


$14,664 


+$4,608" 


+$5,940'" 



The results for the entire 31 -month observation period are slightly higher. The average 
annual rates of earnings over this period were $10,056 for controls and $14,664 for the 
treatment group. The simple-difference estimate of $4,608 per year is similar to the Wave I 
impact estimate of $4,764. The regression-adjusted Wave n impact estimate of $5,940 is 
substantially higher than the Wave I impact estimate. Thus, we can firmly conclude that the 
Enterprise Project had a significant and positive impact on combmed annual earnings from self- 
employment and wage and salary employment. 

Other Demonstration Impacts 

Impacts on Job Creation. In addition to providing employment for the business owner, 
small businesses often generate wage and salary employment for others. This section analyzes 
the demonstrations' impacts on the wage and salary employment of nonparticipants by measuring 
the total employment in businesses operated by treatment group members and compare this total 
with total employment m businesses operated by control group members. The difference 
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between the total employment in treatment group businesses and total employment in control 
group businesses represents an estimate of the demonstration's impacts on the employment of 
nonparticipants. 

When small business owners need employees they sometime hire family members. Small 
businesses also generate employment for nonfamily members. In previous reports we examined 
the employment of family and nonfamily members separately (see Benus, Wood, and Grover, 
1994). Here, however, we analyze the demonstration impacts on total nonparticipant 
employment. Nonparticipant employment includes all family and nonfamily wage and salary 
employees of the business, excluding the demonstration participant him/herself. 

We measure the effect of the demonstration on nonparticipant employment as the 
difference between the mean number of employees per treatment group member and the mean 
number of employees per control group member. Since this analysis is conducted over all 
treatment and control group members (whether they have a business or not), those individuals 
without a business are assumed to have no employees. 

As indicated in Table 8.13, the Enteiprise Project did not significantly mcrease the 
employment of nonparticipants. Indeed during both Wave I and Wave n, the mean number of 
employees per control group member was higher than the mean for treatment group members. 
A closer examination of this result indicates that the high mean for the control group is 
dramatically influenced by a few large businesses (in terms of employment). Specifically, we 
found that a few control group businesses had a large number of employees (over 20 employees 
each). We did not find similar sized businesses among the treatment group. 

Table 8.13 



Job Creation Impacts per Members: Total Employment 
Massachusetts, Cohorts 1 and 2 





Control 


Treatment 


Difference 


Impact 


Wave I 


0.7 


0.3 


-0.4 


-0.4 


Wave n 


1.7 


0.8 


-0.9 


-1.0 
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Impacts on Length of UI Spell. Using data from the UI history files, we developed a 
measure of the number of weeks in the first spell of UI benefits receipt. As shown in Table 
8. 14, on average, controls experienced a first spell of UI benefits of 26.5 weeks while treatment 
group members experienced a 24.5 week first spell. Thus, the impact of the Massachusetts 
Demonstration was to reduce the length of the first UI speU by approximately two weeks. 



Table 8.14 
Length of First UI SpeU 
Massachusetts, Cohorts 1 and 2 



Control 


Treatment 


Difference 


Impact 


26.5 


24.5 


-2.0*" 


-1.8*" 


weeks 


weeks 


weeks 


weeks 



Impacts on UI Benefits. Table 8.15 presents the program impacts on UI benefits 
received in each year of the observation period. In Year 1, control group members received an 
average of $6,845 in UI benefits while treatment group n»,>mbers received an average of $6,120 
(for a simple difference of -$725). The regression-adjusted impact of the demonstration on UI 
receipt in the first year was -$768 (statistically significant at the .01 level), indicating that the 
Massachusetts Demonstration lowered UI benefits receipt during the first year. In subsequent 
years, the demonstration did not have a significant impact on UI benefits receipt. In Year 2, 
control group members received an average of $410 in UI benefits, while treatment group 
members received an average of $307 (a difference of -$103). During the first seven months 
of Year 3 (i.e., months 25 to 31 after the initial UI claim), treatment group members averaged 
$141 in UI benefits and control group members averaged a nearly identical $140. Over the 
entire 31 -month observation period, the demonstration's total impact on UI benefits was -$876 
(statistically significant at the .01 level). 
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Table 8.15 
UI Beneflt Amounts 
Massachusetts, Cohorts 1 and 2 



Time Wrsme 


Control 


Treatment 


Difference 


Impact 


Year 1 


$6,845 


$6,120 


-$725"" 


-$768"' 


(Months 1-12) 










Year 2 


$410 


$307 


-$103 


-$89 


(Months 13-24) 










Year 3 


$141 


$140 


-$1 


-$13 


(Months 25-31) 










Total Wave n 


$7,400 


$6,567 


-$833'" 


-$876"' 


(Months 1-31) 











Impacts Based on Cohort 3 

In Massachusetts, changes in UI ]>olicy altered the implementation of the demonstration, 
especially for Cohort 3.*' As a result. Cohorts 1 and 2 were combined for the main impact 
analysis. Cohort 3 impact results are presented below largely for completeness. Future analysis 
may shed light on how all three cohorts might be pooled to obtain an overall impact estimate for 
the entire Massachusetts sample. 

As described in Chapter 4, in Massachusetts, a total of 701 claimants were randomly 
assigned during the third implementation phase of the Enterprise Project. Due to budgetary and 
other constraints, however, surveys were attempted with only the first 496 sample members to 
be randomly assigned. Furthermore, since the Cohort 3 implementation period was not 
concluded untU 1993, only one round of surveys (Wave I) was possible with this group. 
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•' A complete discussion of these changes was presented in Chapter 4. 

146 

1 vo 



Chapter 8 * Massachusetts Demonstration Impacts 



The 82 percent survey response rate for Cohort 3 yielded a total sample of 405 for the 
impact analysis. The analysis of program impacts that follows is based on this sample (213 
Treatments and 192 Controls). 

As we did above for the Cohort 1 and 2 impact analysis, we first present the self- 
employment impacts, followed by wage and salary impacts, and finally we present combined 
self-employment and wage and salary impacts. After completing this sequence of impact 
analyses, we divide the Cohort 3 sample into two groups: Group 1 is composed of Cohort 3 
sample members who filed their UI claim on or before June 1, 1992; Group 2 is composed of 
Cohort 3 members who filed a UI claim after this date. Claimants who filed their UI claim 
prior to that date were eligible for a maximum of 30 weeks of UI benefits. Those filing after 
that date, were limited to 26 weeks of UI benefits. After splitting the Cohort 3 sample into 
these two subgroups, we re-analyze program impacts for each. 

Self-Employment Impacts 

In this subsection we present the impacts of the Enterprise Project on self-employment 
outcomes on the ftill Cohort 3 sample. As indicated in Table 8. 16, the impact of the Enterprise 
Project on the likelihood of having a self-employment experience during the Wave I observation 
period was positive and significant. The regression-adjusted logistic impact of 20 percentage 
points is higher than the 17 percentage point impact reported earlier for Cohorts 1 and 2. 



Table 8.16 
Self-Employment Impacts 
Massachusetts, Cohort 3 





Control 


Treatment 


Difference 


Impact 


Percent Self-Employed 


45% 


65% 


-H20%'" 


4-20 %•" 


Since Random Assignment 










Time Self-Employed 


1.7 


3.8 


4-2.1"' 


4-2.3'" 


Since Random Assignment 


months 


months 


months 


months 


Percent Self-Employed at Time of 


36% 


57% 


4-21%"' 


4-21 %•" 


Survey 










Self-Employment Earnings 


$1,336 


$3,243 


-l-$l,907" 


4-2,138** 


Since Random Assignment 
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The demonstration impact on time employed in self-employment (2.3 months per year) 
is also higher for Cohort 3 than for Cohorts 1 and 2 (1 .0 month per year). Similarly, the impact 
on the likelihood of employment at the time of the Wave I survey is substantially higher for 
Cohort 3 than for Cohorts 1 and 2 (21 percentage points versus 10 percentage points). 

Perhaps the most dramatic difference, however, is the impact on self-employment 
earnings. Based on the earlier results for Cohorts 1 and 2, we concluded that the Enterprise 
Project did not significantly impact self-employment earnings. The results for Cohort 3, 
however, indicate that the Enteiprise Project had a positive and significant impact on self- 
employment earnings. Specifically, the demonstration increased treatment group members' self- 
employment earnings by $2,138 per year relative to control group members. 

Based on the above impact results, we can conclude that the Cohort 3 self-employment 
results reaffirmed the earlier self-employment results that were based on Cohorts 1 and 2. In 
fact, the Cohort 3 results imply even stronger program impacts than were found in the earlier 
analysis. 

Wage and Salary Impacts 

In Table 8.17 we present the impacts of the Enteiprise Project on wage and salary 
employment outcomes based on an analysis of the Cohort 3 sample. Here we see that the impact 
of the Enteiprise Project on the likelihood of having a wage and salary job during the Wave I 
observation period is negative and significant. This result is in contrast to our earlier fmding 
(based on Cohorts 1 and 2) that the Enterprise Project had no impact on the likelihood of wage 
and salary employment during the Wave I observation period. In fact, the Cohort 3 impact on 
the likelihood of having a wage and salary job during the Wave I observation period is closer 
to the Washington SEED results (see Chapter 7). 
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Table 8.17 
Wage and Salary Employment 
Massachusetts, Cohort 3 





Cnntro! 




Difference 




Percent Employed m W&S 
Since Random Assignment 


64% 


54% 


-10%' 


-10%" 


Time Employed in W&S 
Since Random Assignment 


3.4 
months 


3.0 
months 


-0.4 
months 


-0.5 
months 


Percent Employed in W&S at 
Time of Survey 


52% 


40% 


-12%" 


-12%"' 


W&S Eamingi 

Since Random Assignment 


$8,385 


$5,827 


-$2,558" 


-$2,732" 



The second line in Table 8.17 reports the Enterprise Project impact on time employed 
per year in wage and salary employment. The results indicate that the Enterprise Project has 
a negative but insignificant impact on this outcome. Again, this result differs from the earlier 
(Cohort 1 and 2) results where the impact was positive and significant. 

As indicated in the third line of Table 8.17, the Enterprise Project impact on the 
likelihood of wage and salary employment at the time of the Wave I survey was negative and 
significant. Again, this result differs from the earlier Enterprise Project impacts and supports 
the findings from SEED. 

Even stronger support for the SEED findings is found in the results on wage and salary 
earnings. Based on the Cohort 3 results, the Enterprise Project has a negative and significant 
impact on wage and salary earnings. This result is similar to the SEED results and very 
different from the earlier Enterprise Project results. 

Clearly, the above Cohort 3 results on wage and salary outcomes are perplexing since 
they differ so dramatically from the earlier results (based on Cohorts 1 and 2). Furthermore, 
the above results appear to support the SEED findings (rqjortsd in Chapter 7). 
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Total Employment and Earnings Impacts 

In Table 8.18 we present the Enterprise Project impacts on combined self-employment 
and wage and salar>' employment outcomes based on an analysis of the Cohort 3 sample. A 
comparison of these impact results with the Enterprise Project results presented earlier in this 
chapter (based on Cohorts 1 and 2) as well as with the SEED impact results presented in Chapter 
7 indicates that the Cohort 3 earnings impacts are more similar to the SEED impact results than 
they are to the Enterprise Project Cohorts 1 and 2 results. The Cohort 3 impact estimate on 
total earnings is insignificant, similar to the SEED impact estimate. This is in contrast to the 
large and significant impact on earnings for Cohort 1 and 2 in Massachusetts. 



Table 8.18 

Combined Self-Employment and Wage and Salary Employment 
Massachusetts, Cohort 3 





Control 


Treatmrat 


Difference 


Impact 


Percent Employed 

Since Random Assignment 


89% 


96% 


+7%" 


+7%" 


Time Employed 

Since Random Assignment 


5.3 
months 


7.1 
months 


+ 1.8'" 
months 


-H1.9"" 
months 


Percent Employed at the End of 
Observation Period 


79% 


88% 


+9%" 


+9%" 


Earnings 

Since Random Assignment 


$10,493 


$10,358 


-$135 


-H$134 



Other Impacts 

In Table 8.19, we present the Cohort 3 program impacts on length of the first UI spell 
and on the amount of UI benefits received during the first benefit yeai'. The results for Cohort 
3 are substantially different from the results reported earlier for Cohorts 1 and 2. Whereas for 
Cohoit 1 and 2 we reported that the demonstration significantly reduced the length of the first 
UI spell (see Table 8.14), we now fmd no statistically significant impact. 



150 



Chapter 8 • Massachusetts Demonstration Impacts 



Similarly, where previously we found that the Massachusetts Demonstration significantly 
reduced UI benefits in the first benefit year, we now find no statistically significant impact. 
These UI impact results, together with the other Cohort 3 impact results on employment and 
earnings, suggest a dramatic difference in program impacts between Cohorts 1 and 2 and 
Cohort 3. 



Table 8.19 
UI Impacts 
Massachusetts, Cohort 3 





Control 


Treatment 


Difference 


Impact 


Lengt'u of First 


23.7 


24.4 


0.7 


0.3 


UlSpeU 


weeks 


weeks 


weeks 


weeks 


UI Benefits m 


$6,250 


$6,517 


$267 


$211 


First Benefit Year 











Impacts for Various Claimant Subgroups 

As part of our analysis of program impacts, we examined whether the impacts of the 
Enterprise Project varied by claimant characteristics, including gender, race, site, business 
experience and whether individuals intended to return to their previous employers at the time 
of application to the demonstration. To test for differential program impacts across subgroups, 
we added interaction terms (e.g., treatment status interacted with subgroup characteristic(s)) to 
the basic multivariate regression model described earlier in this chapter. In this section we 
describe the general patterns of the subgroup results for the main outcome measures derived 
from the followup survey data that were statistically significant in both Wave I and Wave II; 
readers interested in additional details can refer to Appendix C. 

First, we examined th(; patterns of subgroup impact differences for the self-employment 
outcome measures. We examined whether any interaction effect was significant in both Waves 
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I and II. The results of this analysis yielded no persistent interaction effect for both observation 
periods. Thus, we conclude that the Massachusetts demonstration impacts on self-employment 
outcomes were widespread and not concentrated on any of the subgroups tested. 

The Enterprise Project did, however, have a differential subgroup effect on one wage and 
salary outcome measures. Specifically, for males, the demonstration reduced the likelihood of 
having a wage and salary job during both observation periods. There were no other significant 
interaction effects thr t persisted in both observation periods. 

Finally, we examined whether there were significant interaction effects on the combined 
self employment and wage and salary employment outcome measures. Again, for these 
outcomes we found that all the impacts are widespread and not concentrated in specific 
subgroups. None of the interaction terms were significant in both Waves I and H. 

Summary 

In this chapter we presented our detailed results concerning the impacts of the Enterprise 
Project on numerous self-employment and wage and salary employment outcome measures, as 
well as on job creation and UI benefits receipt. In this section, we briefly summarize the 
highlights of these findings. In particular, we focus on the main self-employment and wage and 
salary employment and earnings measures based on the followup surveys and on the receipt of 
UI benefits that contribute to the benefit-cost results described in Chapter 10. 

The main impact results of the Enterprise Project are summarized in Table 8.20. As 
these results indicate, the Enterprise Project had positive impacts on only some of the self- 
employment outcomes analyzed. Moreover, the impacts appear to be somewhat greater in Wave 
I than in Wave H. Specifically, in both observation periods, treatment group members were 
more likely than controls to have a self-employment experience (by about 12-17 percentage 
points) and to spend more time per year in self-employment (by 0.8-1.0 months). At the time 
of the Wave I survey, treatment group members were also more likely than controls to be self- 
employed (by 1 1 percentage points). The demonstration, however, did not have a significant 
impact on self-employment earnings during either of the two observation periodc. 
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Consistent with the results on self-employment outcomes, the impacts on wage and salary 
outcomes were greater in Wave I than in Wave H. For example, in Wave I, the Enterprise 
Project increased claimants' time in wage and salary employment by nearly one month per year; 
it also increased the likelihood of working in wage and salary employment at the time of the 
survey by 6 percentage points. In Wave H, however, the impact of the demonstration on these 
outcomes was insignificant. In contrast, the impact of the Enterprise Project on earnings from 
wage and salary employment persisted in both observation periods. Specifically, the 
demonstration significantly increased wage and salary earnings by over $3,000 per year in both 
Waves I and U. 

In the bottom panel of Table 8.20 we present the impacts of the Enterprise Project on 
total employment and earnings outcomes. These combined results are large and consistent across 
both observation periods. Specifically, we find that the program significantly increased the 
likelihood of claimants' finding employment in both observation periods (by 5-11 percentage 
points), increased the number of months worked per year (by 1.9-2.1 months per year), and 
increased the likelihood of being employed by 13 percentage points at the time of the Wave I 
survey and 6 percentage points at the time of the Wave n survey. Perhaps the greatest impact, 
however, is on total earnings. The demonstration increased combined annual earnings by nearly 
$5,000 in Wave I and by neariy $6,000 in Wave H. Thus, it appears that the Enterprise Project 
had a dramatic positive impact on claimants' total earnings. 

We also examined in detail the impacts of the demonstration on UI benefits receipt. 
Although not reported in this table, our results indicate that the demonstration decreased receipt 
of UI benefits by $768 during the initial benefit year. Over the entire 31 -month observation 
period, our results indicate that the Enterprise Project decreased UI benefits received by 
claimants by $876. 
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Table 8.20 
Massachusetts Summary 
Program Impacts 
Cohorts 1 & 2 





Wave I 


Wave II 


ocu-jMupiuyuiciii. 






Percent Self-Employed 
Since Random Assignment 


-f-17%"' 


-f-12%" 


Time Self-Employed 

Sinff TJanHom AQQiPtrniftit 


-f-1.0"' 

months 


-f-0.8' 
months 


Percent Self-Employed at the 

"RnH of 01i^^*rvjitinn PprinH 


-f-11%" 


-f-5% 


Self-Employment Earnings 


-h$l,218 


-f-$l,219 








Percent Employed in W&S 
Since Random Assignment 


+ 1% 


-4% 


Time Employed in W&S 
Since Random Assignment 


-f-0.8" 
months 


-f-0.6 
months 


Percent Employed in W&S at 
the End of Observation Period 


+6%' 


-f-1% 


W&S Earnings 

Sinri* Random Assipnment 


-f- $3,230" 


-f-$3,053" 


XUlitl JCiUipiUjrlllCIlt 






Percent Employed 

Since Random Assignment 


-f-11%"' 


-f-5%" 


Time Employed 

Since Random Assignment 


+2.1"' 
months 


-f-1.9'" 
months 


Percent Employed at the End 
of Observation Period 


-f-13%"' 


-f-6%' 


Total Earnings 

Since Random Assignment 


-f- $4,764" 


-f- $5,940"' 
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COMPARISON OF DEMONSTRATION IMPACTS 



The impacts of the Washington demonstration were presented in detail in Chapter 7; the 
impacts of the Massachusetts demonstration were presented in Chapter 8. This chapter serves 
two related purposes: to consolidate and summarize the impact results that were presented in the 
previous two chapters, and to identify impact differences between the two demonstrations. 

To meaningfully compare program impacts, it is important to take account of the 
different population groups served by the two demonstrations. Only when the. population 
differences are accounted for can we be sure that the impact differences observed actually reflect 
program differences. This chapter describes the procedures that were used to account for the 
population differences and to generate a new Washington analysis Simple. 

Using this new Washington analysis sample, the SEED demonstration impacts are 
reestimated. These reestimated Washington impacts are then compared to the Massachusetts 
impacts. The details of the process used to generate a new analysis sample in Washington are 
described in the following section. Then, using the same format as was used in the previous two 
chapters, we first discuss impacts of the demonstrations on self-employment; next, we discuss 
impacts on wage and salary employment. We then combine self-employment and wage and 
salary employment outcomes and compare the demonstrations' impacts on total employment and 
total earnings. We conclude the chapter with a comparison of impacts on job creation, duration 
of in spells, and UI benefits receipt. 

iDENTinCATION OF A WASHINGTON COMPARISON SAMPLE 

The Washington and Massachusetts self-employment demonstrations served somewhat 
different claimant target groups. The Washington SEED Demonstration was designed to serve 
a broad group of new UI claimants who were not job attached. It did not restrict program 
eligibility to potential applicants based on the number of weeks of UI available to claimants, nor 

155 

173 



Chapter 9 • Comparison of Demonstration Impacts 



did it target individuals who were likely to be long-term unemployed. In conti-ast, the 
Massachusetts Enterprise Project limited entry to the demonstration to those who were eligible 
for at least 26 weeks of UI payments, and used a profiling model approach to restrict the 
program to claimants most likely to exhaust their available unemployment insurance benefits. 
To simulate what the impacts of the SEED Demonstration would have been had the 
Massachusetts targeting philosophy been applied, we imposed similar program eligibility 
requirements to the SEED sample and re-calculated program impacts for the restricted sample 
for comparison with the Massachusetts impact results. Below we describe our approach to 
transform the SEED claimant sample to make it more similar to the Massachusetts sample. 

The process of developing a restricted SEED sample that generally mirrored the target 
group of UI claimants served by the Massachusetts Enterprise Project involved four steps. First, 
we restricted the SEED sample to those who were eligible for at least 26 weeks of UI benefit 
payments. Among the original sample of 752 controls and 755 treatment group members, 
70.5 percent of controls and 70.2 percent of treatments were eligible for at least 26 weeks of 
UI benefits. By imposing this restriction, the original SEED sample of 1 ,507 Claimants was 
reduced to 1,060, with 530 claimants each in the treatment and control groups. 

The second step, similar to the approach used in Massachusetts, involved estimating a 
model to identify claimants likely to be long-term unemployed. To develop su^h a model, we 
used data from a 10 percent random sample of all new claimants in the six SEED sites during 
the demonstration period. Of the over 4,200 claimants in the random sample, we first excluded 
full-referral union members and claimants on standby (i.e., temporary layoff) to incorporate key 
targeting requirements imposed by SEED. We then also excluder^ jiaimants who were not 
eligible for at least 26 weeks of UI benefits, as well <:s those who dirt uot receive at least one 
UI payment. Like the target group for the Massachusetts demonstration, this yielded a sample 
of new claimants who were not job attached, who had received some UI payments, and who 
were eligible for enough UI benefits that they had to be unemployed for a long period to exhaust 
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available UI payments. Among this sample, we then estimated a probit regression model to 
predict the likelihood of exhausting the UI claim.*" 

The third step involved applying the model to calculate the predicted probability of 
exhausting the UI claim for each claimant in the restricted sample. The predicted value of the 
likelihood of exhaustion from the profiUng model ranged among claimants from a low of 
6.8 percent to a high of 58.0 percent; the mean value was 27.2 percent. The distribution of 
predicted values was also very similar for treatment and control group members. 

The fourth and final step involved eliminating from the sample the treatment and control 
group members who were least likely to exhaust their available UI benefits. The approach used 
in Massachusetts estabUshed a threshold value for the predicted probabiUty of exhaustion and 
excluded all claimants who fell below the threshold. As described earUer, this approach resulted 
in excluding only 12 percent from the target pool of interested claimants who had received some 
in and were eligible for at least 26 weeks of UI benefits. To mirror as closely as possible the 
approach used in Massachusetts to select the program's target group, we excluded from the 
SEED sample of 1,060 the 12 percent with the lowest predicted probability of exhaustion, 
resulting in a final trimmed analysis sample of 933. 

Below we presert the impact results based on this trimmed sample. We also compare 
the SEED impact results from the trimmed sample with both the SEED results from the original 
sample and with the Massachusetts demonstration results. 

Impacts on Self-Employment 

In this section, we compare the effects of the two demonstrations on the self-employment 
experiences of program participants. We examine the impact of the two demonstrations on the 



"niie explanatory variables included in the probit model included a set of four dummy variables to capture 
claimant's education level, a set of eight occupation dummy variables based on 1-digit codes of the Dictionary of 
Occupational Titles, wages earned in covered employment in the four complete quarters before filmg the UI claim, 
the wage replacement rate (i.e., the proportion of prior earnings that the weekly benefit amount represents on an 
annual basis), and a set of five site dummy variables. 
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likelihood of entering self-employment, the time spent in self-employment, the likelihood of 
being self-employed at the time of the followup surveys, and the earnings from self-employment. 

Impacts on Likelihood of Self-Employment Experience 

In Table 9.1, we present the impacts of the two demonstrations on the likelihood of 
having at least one self-employment experience during the Wave I and Wave U observation 
periods.'^ The results mdicate that, in both states, treatment group members were significantly 
more likely than control group members to have at least one self-employment experience since 
random assignment. During Wave I, the Massachusetts demonstration increased the likelihood 
of having a self-employment experience by 17 percentage points (or by 49 percent of the control 
group mean). The Washington demonstration's impact was even more dramatic, increasing the 
likelihood by 28 percentage points (or by 80 percent of the control group mean). 



Table 9.1 

Self-Employment Experience Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


35% 


52% 


-hl7%'" 


+ 17%'" 


Washington 


35% 


63% 


+2S%"' 


+2S%"' 


Wave n 










Massachusetts 


47% 


58% 


-hll%" 


+ 12%" 


Washington 


44% 


66% 


-h22%'" 


+22%'" 



"The rciider should note that the observation periods are not identical in the two demonstrations (refer to 
Ch^ter 6 for details). In Massachusetts, the Wave I and Wave n observation periods are approximately 19 months 
and 31 months, respectively. In Washmgton, the Wave I and Wave II observation periods are approximately 21 
months and 33 months, respectively. Some of the impact differences ma> thus be attributable to the observation 
period differences. The time and earnings variables, however, have bean annualized, thus, eliminating the influence 
of different observation periods. 
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In comparison to the Wave I impacts, the Wave n impacts were somewhat smaller. In 
Massachusetts, the impact fell to 12 percentage points (from 17 percentage points); in 
Washington, the Impact fell to 22 percentage points (from 28 percentage points).'^ 

The smaller impacts in Wave n (as compared with Wave I) suggests that the 
demonstrations had two types of effects. The first effect was to expedite entry into self- 
employment among UI claimants who would have entered self-employment even v/ithout the 
demonstration. This first type of effect does not permanently alter the self-employment entry 
rate among UI claimants; it only alters vi^hen these individuals enter self-employment.'' The 
second type of effect was to encourage entry into self-employment among UI claimants who 
might never have considered self-employment without the demonstration. 

One indicator that these two types of effects exist is found in the results presented in 
Table 9.1. That is, when we analyze the impact of the demonstration on the likelihood of 
having a self-employment experience over a relatively short time frame (e.g., Wave I), the 
impact of the demonstration is relatively large since it includes both transitory and permanent 
effects. As the time frame is increased (e.g., from Wave I to Wave H), the transitory effect 
dissipates (since nearly everyone who is likely to enter self-employment has done so) and only 
the permanent effect remains. This phenomenon helps to explain the reduction in impacts (from 
Wave I to Wave 11) identified in Table 9.1. 

We speculate that with nearly a three-year observation period (Wave H), the transitory 
effect of expedited entry has largely dissipated. What remains is the permanent effect of the 
demonstration on entry into self-employment among UI claimants. Thus, the Wave n impacts 



'^The combined effect of excluding claimants who were iligible for fewer than 26 weeks of UI benefits and 
applying the exhaustion algorithm in Washington, can be derived by comparing the results of Table 7.1 with the 
results of Table 9. 1. Using the full Washington sample (Table 7. 1), the Washington demonstration increased the 
likellhcoa of having a self-employment experience since random assignment by 25 percentage points during Wave 
I and 22 percentage points during Wave n. The resulte in Table 9. 1, indicate that the impacts were slightly higher 
when using the more restrictive sample (i.e., those eligible for 26 or more weeks of UI benefits and those most 
likely to exhaust their benefits). Specifically, during Wave I, the impact was 28 percentage points; while, during 
Wave n, the impact was 22 percentage points. 

''Previous analysis (Benus, Wood, and Grover, 1994) showed that SEED expedited entry into self-employment 
by approximately six months, while the Enterprise Project expedited entry into self-employment by about two 
months. 
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may be thought of as the long-term impacts of the demonstrations on the likelihood of self- 
employment. 

Impacts on Time in Self-Employment 

Another dimension of the self-employment experience is the time spent in self- 
employment. In Table 9.2 we compare the demonstrations' impacts on annual time spent in self- 
employment. The impacts are significant in both demonstrations as well as in both observation 
periods. In Washington, the impacts are substantially larger than in Massachusetts. In Wave 
n, for example, the Washington demonstration increased the time in self-employment by over 
two months per year, whereas the Massachusetts demonstration increases the time in self- 
employment by less than one month per year. These results indicate that, over a three-year 
period, the Washington demonstration increased total time in self-employment by about six** 
months; in contrast, the Massachusetts demonstration increased total time in self-employment 
by slightly over two months.'^ 

Impacts on Self-Employment at Time of FoIIowup Surveys 

The above results indicate that both demonstrations significantly increased the likelihood 
of participants' entry into self-employment and the time spent in self-employment. Another 
indicator of program success is the likelihood of being self-employed at some fixed point in time 
following random assignment. In Table 9.3 we present the demonstration impacts on the 
likelihood of being self-employed at the time of the followup surveys. 

As indicated in Table 9.3, both demonstrations had large and positive impacts on the 
likelihood of self-employment at the time of the Wave I survey. In Massachusetts, the impact 
was 1 1 percentage points (38 percent of the control group mean); in Washington, the impact was 



'*This estimate is derived by multiplying the Wave 11 impact estimate r .onths) by 33/12. The Massachusetts 
estimate is derived by multiplying the impact estimate (0.8) by 31/12. 

'■'A comparison of the full-sample results (Table 7.2) with the results in this chapter (based on the likely 
cxiaustecs eligible for 26 or more weeks of UI benefits) indicates that the Washington demonstration had sightly 
smaller impacts on the full sample than it did on the more restrictive sample (Table 9.2). 
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Table 9.2 



Annual Time in Self-Employment Since Random Assignment 





Control 


Treatment 


DiiFference 


Impact 


Wave I 










Massachusetts 


1.4 


2.5 


+ 1.1 


4-1 n*" 

T 1 .U 




months 


months 


months 


months 


Washington 


1.1 


3.8 


+2.7- 


+2.8*" 




months 


months 


months 


months 


Wave n 










Massachusetts 


1.7 


2.6 


+0.9" 


+0.8' 




months 


months 


months 


months 


Washington 


1.1 


3.4 


+2.3*" 


+2.3*" 




months 


months 


months 


months 



Table 9.3 

Self-Employment at Two Different Observation Times 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


29% 


40% 


+ 11%" 


+ 11%" 


Washington 


24% 


43% 


+ 19%"* 


+ 19%"* 


Waven 










Massachusetts 


38% 


43% 


+5% 


+5% 


Washington 


27% 


40% 


+ 13%"* 


+ 13%"* 
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19 percentage points (79 per:»nt of the control group mean). At the time of the Wave n survey, 
however, the Massachusetts demonstration impact declined to only 5 percentage points which 
«^as not significant. The Washington demonstration impact also declined somewhat, but 
remained significant at the .01 level (13 percentage points).** 

Impacts on Self-Employment Earnings 

The results of Table 9.4 indicate that, in Massachusetts, the demonstration did not have 
a significant impact on annual self-employment earnings in either the Wave I or Wave n 
observation periods. In contrast, the Washington impacts were positive and significant in both 
observation periods. 



Table 9.4 

Annual Earnings from Self-Employment Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


$1,404 


$2,916 


+$1,512 


+$1,218 


Washington 


$815 


$2,979 


+$2,164" 


+$2,003' 


Wave n 










Massachusetts 


$1,439 


$2,627 


+$1,188 


+$1,219 


Washington 


$703 


$3,029 


+$2,326" 


+$2,157" 



It is worth noting that the magnitude of the impacts are approximately constant across 
both observation periods. In Massachusetts, for example, the estimated impacts (while not 
significant at conventional levels) were approximately $1,200 per year in both observation 



"Again, the full-sample results (Table 7.3) are slightly smallfr than the results presented in Table 9.3 (based 
on the more restrictive sample of likely exhaustecs eligible for 26 or more weeks of UI benefits). 
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periods. In Washington, the impacts were approximately $2,000 per year during both 
observation periods.'' 

To summarize the above self-employment results, we found that both demonstrations 
significantly increased: the likelihood of having a self-employment experience; the time in self- 
employment; and the likelihood of being self-employed at the time of the Wave n followup 
survey. A comparison of the magnitude of these impacts between the two demonstrations 
indicates that SEED consistently had greater impacts on self-employment outcomes than the 
Enterprise Project. SEED also significantly increased the earnings from self-employment while 
the Enterprise Project did not. 

Impacts on Wage and Salary Employment 

Both demonstrations directed their efforts toward enhancing participants' self -employment 
outcomes. Nonetheless, wage and salary outcomes were also affected by demonstration services. 
In this section, we compare the impacts of the demonstrations on the likelihood of having a wage 
and salary job, the time spent in wage and salary jobs, the likelihood of having a wage and 
salary job at the time of the followup surveys, and earnings from wage and salary jobs. 

Impacts on Likelihood of Wage and Salary Employment Experience 

In Table 9.5, we present a comparison of the demonstration impacts on the likelihood 
of having a wage and salary job during the two observation periods. In Massachusetts, the 
likelihood of having a wage and salary job during both observation periods was not affected by 
the demonstration. That is, we found no statistical difference between the treatment and the 
control groups in the likelihood of having a wage and salary job. In Washington, on the other 
hand, treatm.ent group members were significantly less likely than control group members to 
have a wage and salary job during both observation periods. Tiiese results suggest that while 
the Massachusetts demonstration did not affect the wage and salary employment prospects of 



reader should recall that the Washington impacts are substantially influenced by one outlier. This single 
case increased the Washington estimated annual impacts on self-employment earnings by approximately $800. 
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treatment group members, the Washington demonstration significantly reduced the likelihood of 
wage and salary employment of treatment group members.'* 

Table 9.5 



Wage and Salary Employment Experience Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


64% 


65% 


+ 1% 


1% 


Washington 


80% 


71% 


-9%'" 


-9%'" 


Wave n 










Massachusetts 


75% 


71% 


-4% 


-4% 


Washington 


86% 


78% 


-8%"' 


-7%"' 



Impacts on Time in Wage and Salary Jobs 

In Table 9.6, we present impact estimates on the time spent per year in wage and salary 
employment since random assignment. In Washington, treatment group members spent nearly 
one month less per year in wage and salary employment than controls (in both observation 
periods); in Massachusetts, on the other hand, treatment group members spent nearly one month 
more per year than controls in wage and salary employment (only the Wave I impacts are 
significant).'' 

For Washington, these results are consistent with the fmdings on the likelihood of having 
a wage and salary job experience (see Table 9.5). That is, the Washington demonstration 
significantly decreased both the likelihood of having a wage and salary job and the time spent 



'*A comparison of these results with the Washington full sample results (presentsd in Table 7.5) indicates that 
the full-sample results are slightly smaller. 

*^e Washington full-sample results in Table 7.6 are slightly smaller than the results based on the more 
restrictive Washingtou sample (presented in Table 9.6). 
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Table 9.6 



Time in Wage and Salary Employment Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


3.2 


3.8 


+0.6 


-f-0.8" 




months 


months 


months 


months 


\/ashington 


5.0 


4.2 


-0.9" 


-0.9" 




months 


months 


months 


month 


Wave n 










Massachusetts 


4.1 


4.4 


-f-0.3 


-f-0.6 




months 


months 


months 


months 


Washington 


5.4 


4.6 


-0.8" 


-0.8" 




months 


months 


months 


months 



in such employment during both observation periods. For Massachusetts, on the other hand, tiie 
results are mixed. That is, the Massachusetts demonstration did not affect the likelihood of 
having a wage and salary job in either Wave I or Wave 11; it did however, have a positive 
impact on the time in such employment during Wave I. Thus, in contrast to the negative 
impacts on wage and salary employment outcomes in Washington, the Massachusetts 
demonstration appears to have increased time in wage and salary employment in the short-term. 

Impacts on Wage and Salary Employment at Time of the Followup Surveys 

Table 9.7 presents the impact estimates on the likelihood of wage and salary employment 
at the time of the two followup surveys. In Washington, the demonstration significantly 
decreased the likelihood of wage and salary employment at the time of the Wave I followup 
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survey; at the Wave n survey, however, the impact was insignificant. In Massachusetts, r.here 
was no significant impact at either survey.'"" 



Table 9.7 

Wage and Salary Employment at Two Different Observation Times 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


46% 


52% 


+6% 


+6% 


Washington 


65% 


56% 


-9%"' 


-9%'" 


Wave n 










Massachusetts 


54% 


56% 


+2% 


+ 1% 


Washington 


64% 


60% 


-4% 


-4% 



Impacts On Wage and Salary Earnings 

The impacts on annxial wage and salary earnings during the two observation periods are 
presented in Table 9.8. In Massachusetts, the demonstration significantly mcreased wage and 
salary earnings during both observation periods. Indeed, treatment group members earned 
{^proximately $3,000 more per year from their wage and salary jobs than their control group 
counterparts. In Washington, on the other hand, treatment group members earned approximately 
$2,000 less in Wave I and approximately $1,700 less in Wave n than their control group 
counterparts. This is pertiaps the most dramatic difference in the impacts of the two 
demonstrations. 

One explanation for this dramatic difference in impacts may be found in the design and 
implementation of the demonstrations. As described in earlier chapters, the Washington 



"■^The Wuhington full-sample results, presented in Table 7.7, are slightly smaller than the results presented 
in Table 9.7, 
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Table 9.8 



Earnings from Wage and Salary Employment Since Random Assignment 





Control 


Treatment 


DiinPerence 


Impact 


Wave I 










Massachusetts 


$6,613 


$9,311 


+$2,698" 


+$3,230" 


Washington 


$10,586 


$8,661 


-$1,925 


-$2,064" 


Wave n 










Massachusetts 


$7,797 


$10,119 


+$2,322* 


+$3,053" 


Washington 


$11,041 


$9,323 


-$1,718 


-$1,744* 



demonstration provided the business assistance training and much of the counseling during a 
concentrated, brief period soon after selection into the demonstration. In contrast, the 
Massachusetts demonstration provided business assistance training and counseling over a much 
longer time period. The training modules in Washington were presented during a one-week 
period; in contrast, the Massachusetts training seminars were spread over a twelve-week period. 
Similarly, treatment group members in Washington received on average 1.5 hours of individual 
counseling; in contrast, Massachusetts' treatment group members received, on average, 7.5 
hours of counseling. 

These differences in program design and implementation may partly explain the 
difference between the two demonstrations in wage and salary earnings impacts. That is, the 
longer contact period in Massachusetts between participants and their trainers/counselors may 
have contributed to the more favorable wage and salary earnings outcomes in Massachusetts. 
That is, in Massachusetts, counselors were able to spend more time (than in Washington) with 
their clients to assess whether self-employment remained a viable option after observing the 
client's progress i:i developing the business idea. In some situations, the counselor may have 
helped the client determine that wage and salaiy employment was the best option available. 
While counselors in Massachusetts may not have worked directly on helping treatment group 
members find wage and salary jobs, the substantial time that they spent with their clients may 
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have resulted in improved wage and salary cjtcomes for their clients. Clearly, this 
explanation of the difference in wage and salar>' outcomes t)etween the two demonstrations is 
speculative. It represents only one of many possible explanations. 

Impacts on Total Employment and Earnings 

In the previous sections, we discussed the impacts of the Massachusetts and Washington 
demonstrations on a number of self-employment outcomes as well as on a number of wage and 
salary outcomes. In this section we combine the self-employment and wage and salary outcomes 
and present impact estimates for the combined outcomes. We present the following four impact 
estimates for both demonstrations and both observation periods: the likelihood of having either 
a wage and salary or a self-employment experience; the time employed per year; the likelihood 
of being employed at the time of the followup surveys; and the total earnings per year from 
either type of employment. 

Impacts on Likelihood of Employment Experience 

Table 9.9 presents the estimated impacts of the Washington and Massachusetts 
demonstrations on the likelihood of having had either a wage and salary job or a self- 
employment spell during the two observation periods. The results indicate that, in 
Massachusetts, the demonstration had a positive and significant impact during both observation 
periods. In contrast, the Washington demonstration had a significant impact during Wave I 
only.''" However, given the high level of control group employment in Wave n (especially 
in Washington where 97 percent of the control group had an employment experience during the 
observation period), it is extremely difficult to detect a significant increase. 



""The Washington Wave 11 impacts are similar for both the full-sample (presented in Table 7.8) and for the 
more limited sample of likely exhaustees who were eligible for 26 or more weeks of UI benefits (presented in Table 
<■».?). The Wave I impacts, however, are different in the two samples. The impact for the full-sample is -$1,407 
while the impact for the restricted sample is -$2,064. 

""An examination of Table 7.9 indicates that, for the full Washington sample, the results were also significant 
for Wave I only. 
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Table 9.9 



Combined Employment Experience Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


82% 


92% 


+ 10%"* 


+ 11%"* 


Washington 


93% 


96% 


+3%* 


+3%* 


Wave n 










Massachusetts 


92% 


97% 


. %" 


+5%" 


Washington 


97% 


98% 


+2%' 


+2% 



Impacts on Time in Employment 

Impacts on total time employed are presented in Table 9. 10. In both demonstrations, the 
impacts on time employed are positive and significant. Spiecifically, the Massachusetts 
demonstration increased total time in employment by approximately two months per year during 
both Waves I and H. The Washington demonstration had a similar impact during Wave I, but 



Table 9.10 

Total Time in Employment Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


4.5 


6.5 


+2.0"* 


+2.1- 




months 


months 


months 


months 


Washington 


6.2 


7.4 


+ 1.2- 


+ 1.3- 




months 


months 


months 


months 


Wave n 










Massachusetts 


5.8 


7.4 


+ 1.6"* 


+ 1.9- 




months 


months 


months 


months 


Washington 


6./ 


7.8 


+ 1.2- 


+ 1.1- 




months 


months 


months 


months 
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a smaller impact (approximately one month increase per year) during Wave H. Thus, both 
demonstrations significantly iiioreased the total time employed during both observation 
periods. 

Impacts on Employment at Time of the Followup Surveys 

The impacts on the likelihood of having a wage and salary job and/or being self- 
employed at the time of the follow-up surveys are presented in Table 9.11. These impacts are 
somewhat surprising. At the tune of the Wave I survey, the Massachusetts impact was positive 
and significant, while the Washington impact was insignificant. At the time of the Wave n 
survey, however, the impacts were quite similar: both the Massachusetts and Washington 
impacts were positive and significant. 



Table 9.11 

Employment at Two Different Observation Times 





Control 


Treatment 


DiinTerence 


Impact 


Wave I 










Massachusetts 


68% 


81% 


-rl3%"' 


-f-13%"' 


Washington 


81% 


84% 


-f-3% 


-f-3% 


Wave n 










Massachusetts 


81% 


87% 


-f-6% 


-f-6%' 


Washington 


83% 


88% 


-f-5%" 


-f-5%" 



Thus, the results of Table 9.11 indicate that the Massachusetts impacts dissipated over 
time, while the Washington impacts increased. It is relatively easy to explain the Massachusetts 
pattern of program impacts dissipating over time. It is much more difficult, however, to explain 
the Washington pattern of program impacts. 

One explanation for the Washington result is the significant and opposite effects of SEED 
on the likelihood of self-employment and wage and salary employment. That is, SEED had a 



""The Washiiigton results for the full-sample indicate smaller impacts in time in employment. 
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positive impact on the likelihood of self-employment (Table 9.3) and a negative impact on the 
likelihood of wage and salary employment (Table 9.5). These opposite impacts combined to 
yield an insi;;nificant impact on the likelihood of employment at the time of the Wave I survey. 
At the time of the Wave n survey, however, the strong positive impact on the likelihood of self- 
employment dominated the smaller negative impact on the likelihood of wage and salary 
employment. 

Impact: On Eamixigs 

The earnings impacts of the two demonstrations are presented in Table 9.12. As 
indicated by the results, the impacts are dramatically different for the two demonstrations. In 
Massachusetts, the impacts are positive and significant for both observation periods. Indeed, 
the results indicate that the Enteiprise Project increased treatment group members' earnings by 
more than $4,7(X) per year during Wave I, an increase of over 50% of the control group mean 
earning levels. Ehiring Wave H, the impact was even greater, increasing total earnings by nearly 
$6,000 per year. In contrast, SEED did not have a significant impact on total earnings in either 
observation period. 



Table 9.12 

Combined Earnings from Employment Since Random Assignment 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


$8,483 


$13,151 


+$4,668" 


+$4,764" 


Washington 


$12,256 


$12,943 


+$687 


+$256 


WaveH 










Massachusetts 


$10,056 


$14,664 


+$4,608" 


+$5,940"' 


Washington 


$13,173 


$14,259 


+$1,086 


+$205 



The earnings impact estimates for Massachusetts are dramatic and extremely interesting. 
In Massachusetts, the two earnings impacts (wage and salary earmngs impact and self- 
employment earnings impact) reinforce each other. That is, the Enterprise Project has positive 
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(but insignificant) impacts on self-employment earnings and positive (and significant) impacts 
on wage and salary earnings. The positive total earning impacts in Massachusetts are thus 
supported by both wage and salary and self-employment earnings but dn-'en mainly by the large 
impacts of the Enttrprise ^Project on wage and salary earnings. 

In contrast, tie two earnings impacts in Washington do not reinforce each other, but 
rather, they offset each other. That is, while SEED has a positive and significant impact on self- 
employment earnings, it has an offsetting negative and significant impact on wage and salary 
earnings. The impact on combined earnings is thus canceled and yields insignificant total 
earnings impacts. 

Impacts on Job Creation 

Above, we estimated the impacts of the Enterprise Project and SEED on the employment 
and self-employment experiences of program participants. We found, for example, that, in both 
demonstrations, treatment group members were more likely than control group members to be 
self-employed; we also found that they were employed for a larger fraction of the observation 
period than their control group counterparts. These findings represent the demonstrations' direct 
impacts on the employment experiences of participants. 

The demonstrations may also have indirect impacts on employment. That is, by 
increasing the number of businesses, the demonstration may create new jobs for nonparticipants. 
In the analysis below we consider these new jobs as additional impacts of the demonstrations. 
It is important to note that this impact analysis may overstate the true impact on employment 
since attributing all these jobs to the demonstrations, implicitly assumes that these "new jobs" 
did not displace other jobs. 

For our analysis of the impacts on job creation, we combine the number of family 
members and nonfamily members employed in each business to obtain a total number of 
employees in each business. We measure the effect of the demonstrations on nonparticipant 
employment as the difference between the total number of employees in treatment group 
businesses and the total number of employees in control group businesses.'''* 



Total number of employees includes all family and nonfamily wage and salary employees excluding the 
business owner(s). 
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As indicated in Table 9. 13, in Massachusetts, there was no statistically significant impact 
on nonparticipant employment. In Washington, there was also no significant impact on job 
creation during Wave I. During Wave n, however, SEED had a positive and significant impact 
on job creation. One explanation for this fmding, is that, in Washington, treatment group 
businesses had grown to the point where they were employing nonparticipants. 



Table 9.13 
Job Creation Impacts: Total Employment 





Control 


Treatment 


Difference 


Impact 


Wave I 










Massachusetts 


0.7 


0.4 


-0.4 


-0.4 


Washington 


0.2 


0.3 


+0.1 


+0.1 


Wave n 










Massachusetts 


1.7 


0.8 


-0.9 


-1.0 


Washington 


0.2 


0.4 


+0.3'" 


+0.3"' 



Impacts on UI Outcomes 

In previous chapters we found that the demonstrations also have significant impacts on 
UI outcomes of program participants. In this section we compare the demonstrations' impacts 
on selected UI outcomes, including length of first UI spell and amount of UI benefits received. 

Using data from the UT history files, we developed three main measures of UI benefit 
receipt: number of weeks of the first spell of UI receipt; total amount of UI benefits received 
during the first benefit year; and total amount of UI benefits received during the Wave n 
observation period. 

As shown in Table 9.14, the impact of the Massachusetts demonstration was to reduce 
the length of the first UI spell by approximately two weeks. The impact of the Washington 
demonstration was a reduction of approximately eight weeks. Thus, while both demonstrations 
significantly reduced the length of the first UI spell, the impact in Washington was greater. The 
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greater impact in Washington is largely due to the large number of Washington treatment group 
members who entered self-employment early in their unemployment q)ell following receipt of 
the lump-sum payment. Table 9.14 

Impacts on UI Outcomes 





Control 


Treatment 


Bi^erence 


&npact 


Length of First UI S] 


peU 


Massachusetts 


26.5 . 


24.5 


-2.0"* 


-1.8*" 




weeks 


weeks 


weeks 


weeks 


Washington 


19.3 


11.6 


-l.T" 


-7.6"* 




weeks 


weeks 


weeks 


weeks 


Beneflt Amount in First Year 


Massachusetts 


$6,845 


$6,120 


-$725"* 


-$768*" 


Washington 


$4,361 


$2,555 


-$1,806*** 


-$1,796*" 


(UI only) 










Washington 


$4,361 


$5,787 


-H$l,426*" 


-H$l,451*" 


(UI + Lump-Sum) 










Benefit Amount in Wave n Observation Period 


Massachusetts 


$7,400 


$6,567 


-$833*" 


-$876*" 


Washington 


$5,442 


$3,517 


-$1,925*** 


-$1,946*" 


(Ulonly) 










Washington 


$5,442 


$6,750 


-H$l,308*" 


-H$l,300"* 


(UI + Lump-Sum) 











The second panel of Table 9.14 presents the program impacts on UI benefits received 
during the first benefit year. The Massachusetts demonstration had significant negative impacts 
on UI benefits during the first benefit year. Treatment group members in Massachusetts 
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received approximately $750 less in UI benefits than control group members. Thus, the 
implementation of Enterprise Project saved money for the UI Trust Fund. 

In Washington, the demonstration also had a negative and significant impact on benefits 
receipt (excluding the lump-sum payment). That is, SEED significantly reduced UI benefits 
received by treatment group members relative to control group members (by over $1,800). 
However, taking into account both UI benefit payments plus lump-sum payments yields a very 
different impact estimate. As seen in the Table 9.14, treatment group members on average 
received approximately $1,450 more than control group members in total benefits (i.e., UI 
benefits plus lump-sum payment) during the first year. Thus, whereas the Massachusetts impact 
estimate indicates a reduction in total payments to program participants, the Washington impact 
estimate indicates an increase in total payments to program participants in the first benefit year. 

An analysis of benefits received during the Wave n observation period reveals similar 
results. In Massachusetts, treatment group members continued to collect less than their control 
group counterparts; indeed, for the entire Wave n period, the impact grew to -$876. In 
Washington, there was a similar pattern, indicating that treatment group members collected less 
UI benefits than their control group counterparts in the years following their program 
participation. Specifically, without the lump-sum payment, the demonstration impact was a net 
reduction in UI benefit payments of -$1,946; including the lump-sum payment, the impact 
became a net increase in total payments of $1,300. 



'"5 Approximately 6096 of the Washington treatment group received a lump-sum payment. The average lump' 
sum payment received was $4,225. 
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BENEFITS AND COSTS 
OF THE SELF-EMPLOYMENT DEMONSTRATIONS 



In previous chapters we presented impact estimates for the Massachusetts and Washington 
UI Self-Employment Demonstrations. The impact results presented in Chapters 7, 8, and 9 
indicate that both demonstrations had beneficial effects on a number of outcomes. Impact 
estimates, however, even when they indicate beneficial program effects, are not sufficient to 
conclude that a program is worthwhile for society. To conclude that a program is worthwhile 
for society, the beneficial effects of a program must outweigh its costs. 

In this chapter we address the question of whether or not UI self-employment programs 
are worthwhile for society by comparing the net benefits of the Massachusetts and Washington 
self-employment demonstrations with their net costs. The Washington net benefits were 
presented in Chapter 7 and the Massachusetts net benefits were presented in Chapter 8. The net 
costs are derived from administrative records and from cost data collected from program 
administrators. 

We begin this chapter by discussing the conceptual framework for the benefit-cost 
analysis. We describe how net social value is calculated from each of four perspectives: that 
of the participants, nonparticipants, the government sector, and society as a whole. Following 
this conceptual discussion, we present the estimated benefits and costs for each of the 
demonstrations. The chapter then concludes with a discussion of the implications of the 
fmdings. 
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Benefit-Cost Analysis 

We measure social benefits and costs as changes from what would have occurred in the 
absence of the demonstration. Social costs are defined as the additional resources necessary to 
provide demonstration services, plus any adverse effects on participants or others that would not 
have occurred in the absence of the demonstration. Similarly, social benefits are defined as the 
value of all beneficial effects that would not have occurred in the absence of the demonstration. 
Net social benefits are the sum of social benefits and costs. If the incremental social benefits 
of a demonstration exceed its incremental social costs, net social benefits are positive; if costs 
exceed benefits, net social benefits are negative. 

Social programs are generally viewed as cost-effective if they yield positive net benefits 
to society as a whole. In assessing the desirability of a program from a policy perspective, 
however, it is also important to take into account the distributional consequences of the program 
— i.e. , the benefits and costs to particular groups within society. Thus, we measure the benefits 
and costs of the demonstration from four different perspectives ~ those of the participants, 
nonparticipants, the government sector, and society as a whole. 

Exhibit 10. 1 shows how the anticipated demonstration effects can be classified within the 
social benefit-cost accounting framework from the perspectives of three of the four major groups 
of interest. In this exhibit, a plus (+) denotes a benefit from the perspective of the group in 
whose column it occurs, a minus (-) denotes a cost to that group, and a zero (0) indicates that 
this item is neither a benefit nor a cost to that group. Each effect of the demonstration is 
classified as a benefit, a cost, or a transfer. "Transfers" are effects that represent a benefit to 
one group in society and an exactly offsetting cost to another group. For example, the increased 
taxes paid by program participants on their higher earnings are a cost to the participants. These 
costs, however, benefit nonparticipants, resulting in no monetary gain or loss to society as a 
whole. 

There are three principal anticipated benefits to participants: earnings gains, improved 
psychological well-being as a result of being self-employed, and enhanced community economic 
development. These anticipated benefits are indicated by pluses in the participant column in 
Exhibit 10. 1 , In previous chapters, we decomposed program impacts on earnings by considering 
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Exhibit 10.1 

Social Benefits and Costs of the Self-Employment Demonstrations 





Pmpective: 




Society ~ 


F^rtteipants + 


Nonparticipants 


Benefits 








Earnings and asset eains 


-1- 


-1- 


0 


Wage and salary 
earnings 

Self-emnlovment 
earnings 








Improved psychological 
well-being 


-1- 


-1- 


0 


Communitv economic 
development 


-1- 


-1- ■ 


-1- 


Costs 








Demonstration operations 




0 




Administration 
Services 








Transfers 








UI benefits (including 
lump-sum payment) 


0 


+ 




Reduced transfer benefits 


0 




-»- 


Increased tax payments 


0 




-1- 



self-employment earnings and wage and salary earnings separately. For the benefit-cost 
analysis, this decomposition of total earnings is not relevant ~ only impacts on total combined 
earnings are considered. 



178 



Chapter 10 • Benefits and Costs of the Self-Employment Demonstrations 



Of the above three principal anticipated program benefits, only community economic 
development is expected to benefit nonparticipants. That is, nonparticipants are expected to 
benefit from increased employment opportunities and increased tax revenues. The other two 
anticipated program benefits, earnings gains and improved psychological well-being of 
participants, are not expected to affect nonparticipants. 

Since none of three principal anticipated benefits to participants represent a cost to any 
other group, they are also benefits to society as a whole. This is indicated in Exhibit 10.1 by 
pluses in the society column. 

The principal social costs of the demonstration are the costs of demonstration operations - 
- program administration and provision of services. These costs are borne by nonparticipants 
and represent a cost to society as a whole. 

The principal transfers expected to -v-a u from the demonstration are the UI benefits 
(including the lump-sum payment), reduced public transfers to participants, and increased tax 
payments by participants. Each of these represents a benefit or cost to participants and an 
exactly offsetting cost or benefit to nonparticipants, and therefore neither a benefit nor a cost 
from the standpoint of society as a whole. While transfers do not affect the net social benefit 
of a program, it is nonetheless important that they be measured in order to assess the 
distributional consequences of the program. 

To summarize, from the perspective of program participants, the key benefits of a self- 
employment program are the increased earnings, improved psychological well-being from being 
self-employed rather than working for someone else, the enhanced community economic 
development, and the UI benefits (including the lump-sum payment). On the other hand, the 
key costs (from the participants' perspective) are the reduced transfer payments received and the 
increased tax payments. 

From the perspective of nonparticipants (i.t., taxpayers), the key benefits are the 
enhanced community development, reduced transfer payments, and increased tax payments. The 
key costs, on the other hand, are the expenditures for demonstration operations and the self- 
employment allowance payments. 

Up to this point, we have treated all taxpayers indistinguishably as nonparticipants. 
Exhibit 10.2 shows the distribution of the benefits and costs of the demonstration within the 
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Exhibit 10.2 

GoTeniment Benefits and Costs of the Self-Employment Demonstrations 







Perspective: 






Govermnent ~ 
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+ 


+ 
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Reduced transfers 


+ 


0 


+ 
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Costs 










Demonstration operations 






0 


0 


• Administration 

• Services 










UT benefjts (including lump- 
sum payment) 




0 




0 



government sector. As shown in the exhibit, the government sector can be split into the U.S. 
Department of Labor (DOL), the UI Trust Fund, and all other government agencies. The 
principal benefits of the demonstration to the government are increased taxes and reduced 
transfers as a result of participants' enhanced economic well-being.** 

Both State and Federal taxpayers will benefit from increased FUTA (Federal 
Unemployment Tax Act) taxes, income taxes, sales taxes, social security contributions, and other 
revenues generated by increased business activity. State and Federal taxpayers will also benefit 
from reduced outlays for UI benefits, welfare, and other transfer payments. 



'°* Note that outcomes that were designated transfers from ». social perspcciive are classified as benefits when 
viewed from the government perspective, because the offsetting costs to participants are outside the govenunent 



sector. 
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The principal costs of the demonstration to the government are for demonstration 
operations, including both program administration and services to participants. It is assumed that 
these costs are borne by DOL. The UI benefits (including the lump-sum payment) represent 
another principal cost. In our benefit-cost analysis we allocate these costs to the UI Trust 
Fund.'°^ That is, we assume that, in an ongoing program, these payments will be drawn from 
the UI Trust Fund. This assumption has recently been bolstered by the provisions of the North 
America Free Trade Agreement (NAFTA) Implementation Act (P.L. 103-182) that authorize 
States to implement self-employment assistance programs. 

. One requirement of this legislation (Section 507) is that "the program does not result in 
any cost to the Unemployment Trust Fund (established by section 904(a) of the Social Security 
Act) in excess of the cost that would be incurred by such State and charged to such Fund if the 
State had not participated in such program." Thus, the Act stipulates that benefit payments to 
UI claimants receiving self-employment assistance are to be paid from the Unemployment Trust 
Fund. The total amount of these payments, however, may not exceed the total amount that 
would have been paid in regular UI benefits. 

As noted above, the net benefit of the demonstration to any group is the sum of the 
benefits and costs accruing to that group. In order to derive all-inclusive measures of net 
benefits, it would be necessary not only to measure all of the benefits and costs of the 
demonstration, but also to value them in commensurate units, so that they can be summed. The 
most natural units in which to measure and sum benefits and costs are, of course, dollars. 
Unfortunately, it may not be possible to measure some demonstration benefits and costs at all, 
and it will not be possible to value all those that can be measured in monetary terms. For 
example, improved psychological weU-being can at best be measured in nonmonetary terms. 

Our approach to benefits that cannot be measured in monetary terms will be as foUows. 
We will first estimate net monetary benefits - the sum of aU benefits and costs that can be 
measured in monetary terms. If net monetary benefits are positive - i.e., if monetary benefits 



It should be noted that in the UI Self-Employment Demonstrations, these costs were not borne by the UT 
Trust Fund. Rather, in Washington these payments were funded through DOL research funds. In Massachusetts, 
payments were paid out of the UI Trust Fund; however, Massachusetts was required to reimburse the Fund if it 
incurred "excess costs" (see Chapter 2). 
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exceed monetary costs ~ there is no need to value nonmonetary benefits in dollar terms, since 
they can only increase net benefits, whatever their value. (Note that no nonmonetary costs are 
anticipated.) Alternatively, if monetary costs exceed monetary benefits, then the excess of 
monetary costs over monetary benefits ("net social cost") is a measure of how valuable the 
nonmonetary benefits would have to be for the program to be worthwhile -- i.e., for overall net 
benefits to be positive. 

Measurement Of Outcomes 

As discussed in the previous section, the principal benefits, costs, and transfers expected 
to result from the demonstration are earnings gains, improved psychological well-being of 
participants, enhanced economic community development, the costs of demonstration operations, 
reduced transfers from UI and other government programs, and increased tax payments. In this 
section, we discuss how we measure each of these demonstration benefits and costs. 

As noted at the outset, social benefits and costs are defmed as changes from what would 
have occurred in the absence of the demonstration. Thus, in general, the benefits and costs of 
the demonstration will be measured by the difference between the outcomes of the treatment 
group and those of the control group, which represents what would have happened to the 
tntatment group in the absence of the demonstration. Below, we first discuss how the benefits 
are defmed and measured and then we discuss the definition and measurement of costs. 

Program Benefits 

As discussed above, the main social benefits of the self-employment demonstrations are 
earnings gains, improved psychological well-being, and community economic development. Of 
these three benefits, earnings gains are the most straight-forward to measure. The other two 
benefits ~ psychological well-being and community economic development ~ are much more 
difficult to measure since they are not readily quantifiable in monetary terms. 

Measurement of psychological well-being represents an extremely difficult methodological 
challenge. We evaluated differences in overall job satisfaction between the self-employed and 
those in wage and salary employment on the basis of responses to job satisfaction questions that 
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were incorporated into the surveys. We asked both the self-employed and those in wage and 
salary employment whether or not self-employment is "better" than working for someone else. 
The results, presented in an earlier report (see Benus, Johnson, and Wood, 1993) showed that 
treatment group members had greater job satisfaction than controls. 

We have not attempted to quantify this nonpecuniary impact of the demonstration more 
precisely (e.g., in monetary terms). As stated earlier in this chaptfc. if the net monetary 
benefits of the demonstrations are positive, then the addition of this nonpecuniary effect will only 
enhance net benefits and not affect the assessment of whether the program is worthwhile from 
society's perspective. 

The measurement of total community economic development effects of the demonstration 
is beyond the scope of this evaluation. As a result, we did not attempt to measure the effect of 
the demonstrations on local economic conditions or on other community-wide outcomes. We 
did, however, measure several key indicators of the degree to which the demonstration 
succeeded in enhancing economic development. For example, we measured the net number of 
new businesses created as a result of the demonstration - i.e., the difference in new business 
starts between the treatment and control groups. We also measured the net employment gains 
of nonparticipants that were attributable io the demonstration. 

As in the case of psychological well-being, we did not attempt to quantify the monetary 
value of these demonstration impacts. If the net monetary benefits of the demonstrations are 
otherwise positive, then the addition of community economic effects will only enhance net 
benefits. 

Earnings gains represent the main benefit that we were able to measure in monetary 
terms. In previous chapters, we isolated the separate demonstration impacts on self-employment 
earnings and on wage and sal?x> earnings in order to gain insights into demonstration effects. 
For the cost-benefit analysis, however, the origin of the earnings gain is less important than the 
magnitude of the total gain. As a result, we use total earnings in the cost-benefit analysis.'"* 
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"* Our measure of earnings gain does not include the value of fringe benefits. While it is relatively straight 
forward to impute fringe benefits for wage and salary earners, imputation of fringe benefits for the self-employed 
is more problematic. Inasmuch as self-employment earnings are an important component of the total earnings gain, 
we chose not to impute the value of fringe benefits. 
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In 'ihe cost-benefit analysis, we use the point estimate regardless of its statistical 
significance as the best estimate of program impacts. The rationale for this is as follows. In 
earlier chapters, when we were testing hypotheses about the program, impact estimates that were 
statistically insignificant resulted in inconclusive tests. In the benefit-cost analysis, however, v/e 
are not testing a hypothesis but rather we are trying to provide the best estimate of the net 
benefits of a particular program or service. In this environment, the actual point estimates 
represent the best measures of benefits and costs. 

We use the Wave II impact estimates for the cost-benefit analysis for several reasons. 
First, since benefits are likely to accrue well after program services end, Wave I impacts may 
not fully capture the total program impacts, indeed, for some participants, especially those in 
Massachusetts, Wave I results may reflect less than one year of experience after the end of 
program services. At the same time, extrapolating the results beyond the available observation 
period is dangerous - especially in Massachusetts, where the economy and UI regulations 
changed dramatically during this period. As a result, we cannot be confident in extrapolating 
the impacts beyond the observation period using our current data. 

It should be noted, however, that there is some evidence that program benefits may grow 
over time. For example, in Massachusetts, earnings gains in Wave n were greater than in 
Wave I. In Washington, we observed an increased impact on job creation in Wave H. Thus, 
truncating the observation period at less than three years after random assignment may understate 
long-term program benefits. 

Based on this analysis and on data availability, we determined that the most appropriate 
time frame for the cost-benefit analysis is the longest observation period available in our data. 
Thus, we conclude that the Wave n impact estimates are the most appropriate estimates of 
earnings gains for the cost-benefit analysis. 

As indicated in Chapter 7, the Wave n annualized impact of the Washington 
demonstration on total earnings was $289. The annualized Wave n impact of the Massachusetts 
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demonstration on total earnings (presented in Chapter 8) was $5,940.'°^ In contrast to the 
Washington estimates, the Massachusetts estimates were statistically significant. 

The measure of earnings that is appropriate for the benefit-cost analysis, however, is not 
annualized earnings, but rather, total earnings over the entire observation period. To derive total 
earnings over the entire observation period, we reestimated the earnings regressions with total 
earnings as the dependent variable. The results of this reestimation of impacts on total 
earnings yields the following results. 



Table 10.1 
Demonstration Total Earnings Impacts 





Washington 


Massachusetts 


Waven 


$1,093 


$14,859'" 



As indicated in Table 10.1, the Wave n (33-month) impact estimate in Washington is $1,093; 
the Massachusetts impact estimates is $14,859. The Massachusetts impact estimate is 
statistically significant. 

Program Costs 

The operational costs of the demonstration include the administrative costs of outreach, 
intake, and program management, and the costs of providing training and counseling to 
demonstration participants. We recognize that to accurately reflect the social cost of 
demonstration activities, the costs must include not only direct labor costs of demonstration staff, 
but also the associated fringe benefits and administrative overhead, and such nonlabor costs as 



The Massachusetts cost-benefit analysis is based on impact estimates derived from the Cohorts 1 and 2. 
Impact estimates derived from Cohort 3 are used later in this chapter to assess the sensitivity of the Massachusette 
cost-benefit analysis results. 

As in previous chapters, fringe benefits are not included in the earnings figures and total earnings are deflated 
to 1990 dollars using a CPI index. 

185 



o 203 
ERIC 



Chapter 10 • Benefits and Costs of the Self-Employment Demonstrations 



rent, utilities, and supplies. We did not, however, attempt to measure non-labor costs."' 
Instead, we focused our efforts on measuring the direct labor costs attributable to the 
demonstration. In addition, to appropriately measure the costs of an ongoing program, our 
measure of direct operational costs does not include several types of cost that are artifacts of the 
evaluation: the evaluation contractor's costs; any costs incurred by demonstration staff for 
evaluation data collection activities; and the costs of outreach and intake associated with the 
control group. We turn now to the measurement of program costs in each of the 
demonstrations. 

Massachusetts Enterprise Project Costs 

As discussed above, a number of costs are associated with an ongoing self-employment 
program. For example, there are administrative costs associated with operating an ongoing self- 
employment assistance program. An even larger component of total cost is the cost of providing 
training and counseling assistance to participants. Finally, there are the UI benefit payments that 
are paid to program participants. 

Inasmuch as the Massachusetts Enterprise Project was an experimental demonstration, 
we use the operating costs of the demonstration only as a starting point and make adjustments 
to simulate the likely cost of an ongoing program. To simulate the administrative and service 
costs of an ongoing program, we eliminate those costs that are specific to the demonstration and 
would not be associated with an ongoing self-employment training program. For example, 
activities associated with random assignment and demonstration data entry are not included in 
the estimate of program costs. 

Administrative Costs. Since our goal is to measure steady-state operating costs, rather 
than costs associated with program start-up, we collected administrative cost data late in the 
demonstration implementation (October 1992). October 1992 represents a period (during the 
third implementation phase) when program administrators and service providers had substantial 
experience with program operations and no new program adjustments were being implemented. 



Since we did not impute fringe benefits in the measurement of program benefits, we do not impute fringe 
benefits in the measurement of program costs. 
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For a four-week period during October 1992, each local office staff member involved 
with the demonstration was asked to complete a Staff Time Allocation Form (STAF). The form 
was designed to collect information on time spent conducting Information Sessions, conducting 
Enrollment Interviews, reviewing applications, discussions with central office staff, answering 
claimants' questions, and so forth. Similarly, a STAF was designed to collect information from 
the Enterprise Project Central Office Staff. The latter form collected information on program 
administrative activities. Time spent on administrative activities that are not associated with an 
ongoing program (e.g., preparing demonstration reports) was not included in our estimate of 
program costs. 

To obtain administrative cost per participant, we divided the total cost of local office 
operations and Central Office operations (during this four-week period) by the average number 
of treatment group members served during a four-week period in phase three of the 
demonstration. 

Using the above procedures, we derived an estunate of administrative cost per participant 
for the Massachusetts Enterprise Project. We estimated a cost of $56 for local office operations 
and a cost of $178 for Central Office operations. We now turn to the procedures used in 
deriving the costs of program services. 

Service Costs. Estimating service costs in the Massachusetts Enterprise Project are 
relatively easy to estimate since aU training and counseling services were provided by outside 
vendors. That is, after 1990, training and counseling services were provided by the vendors 
who were selected through competitive bidding. Thus, the service costs of the program during 
the second and third phases , reflect the market cost of these services. Combining all vendor 
expenditures for Cohorts 2 and 3, and dividing this total by the number of treatment group 
members in Cohorts 2 and 3, we obtain an average cost per treatment group member of $782. 
This cost includes all training and counseling services provided to Massachusetts demonstration 
participants. 

UI Payments. The results of our analysis of UI benefit payments (Chapter 8) indicate 
that the demonstration had a negative impact on these benefits during the first benefit year. That 
is, treatment group members collected less total benefits than control group members in the first 
benefit year. The magnitude of this reduction in UI benefits was -$758. 
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An analysis of UI benefits beyond the first benefit year indicated that savings continued 
to accrue to the UI Trust Fund beyond the first year. That is, in the second and third benefit 
years treatment group members continued to collect less UI benefits than control group 
members. The impact in the second benefit year was -$140 (insignificant). For the 31 -month 
period following random assignment as a whole, the estimated impact of the demonstration on 
UI benefits was a reduction of $935."^ We use the latter figure in the cost-benefit analysis. 

Washington SEED Costs 

The primary coists of operating the Washington SEED program on an ongoing basis are 
the costs of the business development specialists who provide the busmess assistance services, 
the costs of Job Service Center staff labor (for conducting the Information Sessions) and the 
costs of central office staff labor for performing project oversight and administrative duties (e.g. , 
sending invitations for the Information Sessions, scheduling business training, reviewing program 
applications, administering the lump-sum payment process). There are also UI payments 
(including lump-sum payments) which, although a transfer and neutral from society's perspective 
overall, affect the distribution of benefits and costs between participants and nonparticipants. 
Below we outline our approach to measuring program costs for the Washington SEED project, 
and describe our cost estimates. 

Administrative Costs. To measure the administrative costs of operating SEED, we 
relied on summary program cost records maintained m the central office, adjusted to cover only 
the functions required to operate the program on an ongoing basis. Because the 
research/evaluation and random assignment activities that would not be conducted as part of an 
ongoing program were exclusively performed by central office staff, we only had to make 
adjustments to the costs recorded for the central office. After making these adjustments, we 
estimate the -central office staff administrative costs of operating a program like SEED on an 
ongoing basis to be $128 per participant. Our estimate of the cost per participant for local JSC 
staff is much less, $17. This is not surprisuig, given the limited role of JSC staff in the daily 
operations of SEED. 



To derive this impact estimate, we estimated a regression where the dependent variable was UI benefits 
collected during the 31 -month period following random assigmnent. 
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Service Costs. Through an interagency agreement with the Department of Employment 
Security, the Business Assistance Center, a division of the Washington State Department of 
Trade and Economic Development, was responsible for recruiting and hiring Business 
Development Specialists (BDSs) who provided the business assistance ser,ices for the SEED 
Demonstration. Information on the costs incurred in providing SEED business assistance 
services was maintained by the Small Business Development Center (SBDC) of Washington State 
University. These costs included the salaries and fringe benefits paid to the BDSs who provided 
the business training workshops and counseling services to SEED treatment group members in 
the six sites. Combining all costs for business development specialist services across the six 
sites and dividing by the number of treatment group members results in an average cost per 
participant of $252."^ 

UI Payments. An important aspect of the distributional impacts of the program is the 
impact of SEED on UI payments. As described in Chapter 7, after including the lump-sum 
payment as part of total UI payments, we estimate that SEED increased total payments by 
+$1,098 during the first benefit year. 

An analysis of UI benefit payments beyond the first benefit year indicated that there were 
very small differences between the treatment and control groups in benefit payments received 
during subsequent years. Specifically, the point estimate indicates that SEED reduced UI 
payments received after the initial benefit year by -$85, although the effect was not statistically 
significant. For the entire period following random assignment, the estimated impact of the 
SEED Demonstration on total benefit payments was +$1,013. This is the figure used in the 
cost-benefit analysis. 

Comparison of Costs Between Massachusetts and Washington Programs 

As the analysis described above indicates, the costs of the self-employment programs in 
Massachusetts and Washington differ considerably. The costs differ both in terms of the overall 
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The estimated service costs of SEED are considerably lower than the corresponding service costs for the 
Massachusetts Enteiprise Project. This may partly be due to the fact that in Washington some services (e.g., 
secretarial services, office space, and telephone) were made available through Community Colleges or other 
organizations. 
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amounts and in terms of their distribution. In Table 10.2, we provide summary information on 
the costs of the two progiams. 

Table 10.2 

Comparison of Costs Between Massachusetts and Washington 
(Dollars Per Claunant) 





Perspective 




Participants 


Nonparticapants 


. v i Soeiefy 


Massachusetts 








UI Payments 


-$935 


+$935 


0 


Demonstration Costs 








Local Office 
Labor 


0 


-$56 


-$56 


Central Office 
Labor 


u 




TO 

/o 


Business Services 






-lb loZ 


Total 


-$935 


vox 




• 

Washington 








UI Payments 


+$1,013 


-$1,013 


0 


Demonstration Costs 








Local Office 
Labor 


0 


-$17 


-$17 


Central Office 
Labor 


0 


-$128 


-$128 


Business Services 


0 


-$252 


-$252 


Total 


+$1,013 


-$1,410 


-$397 
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As shown in the final column, the costs of each component are much greater for the 
Massachusetts Enterprise Project than for SEED. The most striking difference concerns the 
costs of business services, for which we estimate a per participant cost of $782 for the 
Massachusetts program, as compared to only $252 for SEED. Although part of this differences 
may be due to differences in the underlying cost data available in the two states and the fact that 
the vendor cost data in Massachusetts may more closely reflect true resource costs, it is also 
likely that the large cost differences between the two States reflect the higher costs of providing 
more extensive business counseling services. 

As described in Chapter 5, treatment group members in Washington received only 1.5 
hours of counseling on average, as compared to 7.5 hours per person in Massachusetts. Also, 
the busmess training sessions in Washington were provided shortly after random assignment, 
whereas the business training workshops in Massachusetts were offered over a period of roughly 
9-12 weeks. The large differences in the costs of business services per participant in the two 
programs in large part likely reflect the higher costs of providing more business counseling and 
training services over a longer period. 

The results reported in Table 10.2 indicate that total social costs per participant in the 
Massachusetts program were slightly over $1,000. In contrast, per participant costs in 
Washington were just under $400, or less than one-half of the Massachusetts costs. 

The distributional effects are also dramatically different. The overall cost per participant 
in Massachusetts of -$1,016 is comprised of -$935 from the perspective of participants and -$81 
from nonparticipants. The preponderance of the costs falling onto participants is due to the 
reduction in Ul payments they receive. In contrast, the overall cost per participant of -$397 in 
Washington can be decomposed into +$1,013 from the perspective of participants (due to the 
lump-sum payment), and -$1,410 from the perspective of nonparticipants. 

Net Benefits and Costs 

In this section, we present the net benefits and costs of the Massachusetts and Washington 
demonstrations. The estimates presented in this section are our best estimates of the benefits and 
costs of the two demonstrations. In the next section of this chapter, we present a numbe'- of 

191 



Chapter 10 • Benefits and Costs of the Self-Employment Demonstrations 



alternative estimates of net benefits and costs to assess the sensitivity of our estimate to 
alterr.ative assumptions. 

From all the perspectives examined (i.e., participants, nonparticipants, and society), the 
Massachusetts demonstration had positive net benefits. The Washington demonstration, on the 
other hand, had positive net benefits only from the perspective of participants and from the 
perspective of society as a whole. From the perspective of nonparticipants, the Washington 
demonstration had negative net benefits. We review each of these results below. 

Massachusetts 

Table 10.3 presents the estimated benefits and costs in Massachusetts. As indicated 
above (Table 10.1), in Massachusetts participants experienced a $14,859 increase in total 
earnings during the 31 -month observation period relative to the control group. This estimate of 
earning gains does not include the value of fringe benefits that may be associated with increased 
earnings."* We have chosen to present earnings without this imputation for fringe benefits 
since imputing fringe benefits, especially for the self-employed, is subject to a great deal of 
uncertainty. Our estimates of earnings gains may, thus, be viewed as conservative estimates of 
the actual gain. 

Over the same period, participants paid $2,229 in additional taxes"* and receival $876 
less in UI benefits than their control group counterparts. Summing all these gains and losses, 
the net benefit was $11,754 from the perspective of the average participant. 

The increased taxes paid by participants ($2,229) as well as the reduced UI benefits 
received by participants ($876) are both benefits to nonparticipants. The cost of providing self- 
employment training services, however, represent a cost to nonparticipants. Summing the 
benefits and costs to nonparticipants, yields a net benefit of $2,089. 

For society as a whole, the principal benefit of the program is the earnings gain. The 
principal cost, on the other hand, is the cost of implementing the program. Given the substantial 
earnings gain and the relatively modest program cost, the net benefit to society is $13,843. 



Studies of employee fringe benefits by the U.S. Chamber of Commerce and others have estimated that the 
value of these benefits are approximately 25 percent of total wage and salary outcomes. 

Increased taxes on earnings are estimated as 15 percent of earnings gains. 
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Table 10.3 

Massachusetts 
Benefit-Cost Analysis 
(Dollars per Claimant) 



Benefits and Costs 


Perspective 


Participants 


Nonparticipants 


Society 


Earnings 


$14,859 


$0 


$14,859 


Taxes 


-2,229 


+2,229 


0 


UI Payments 


-876 


+876 


0 


Demonstration Costs 








Local Office Labor 


0 


-56 


-56 


Central Office Labor 


0 


-178 


-178 


Services 


0 


-782 


-782 


NET BENEFITS 


$11,754 


$2,089 


$13,843 



In Table 10.4, we assess the benefits and costs of the Massachusetts Enterprise Project 
from the government's perspective. We decompose total government into the Labor Department 
and all other government agencies."* As indicated in Table 10.4, the Massachusetts 
Enterprise Project had positive net benefits from total government's perspective as well as from 
DOL's perspective from other agencies' perspective. It should be noted, however, that the 
positive net benefit to DDL is quite small ($16)."'' In contrast, the net Ijenefit to total 
government is quite substantial ($2,089), resulting largely from the increased taxes generated 



"* In our analysis we do not isolate the benefits and costs to the UI Trust Fund; rather, we incorporate these 
benefits and costs into the Labor Department perspective. 

The small positive net benefit to DOL is subject to a great deal of uncertainty and could easily have been 
EtgaJive under slightly different assumptions. For example, it is necessary to apportion the additional tax rr-venue 
beiwctu the Department of Labor and the rest of govenunent. We used LOS percent of total wage and salary 
income as the portion of tax payments which go to the UI system. This average rate for the U.S. was reported m 
USDOL's Handbook of Unemployment Insurance Financial Data. 
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Table 10.4 

Massachusetts 
Benefit-Cost Analysis 

Government Perspective 
(Dollars per Claimant) 



BeneOts and Costs 


Perspective 


Labor 
Department 


Other 
Government 


Total 
Government 


Earnings 


$0 


$0 


$0 


Taxes 


+ 156 


+2,073 


+2,229 


UI Payments 


+876 


0 


+876 


Demonstration Costs 








Local Office Labor 


-56 


0 


-56 


Central Office Labor 


-178 


0 


-178 


Services 


-782 


0 


-782 


NET BENEFITS 


+$16 


$2,073 


$2,089 



by the higher earnings of the treatment group members relative to the control group members. 

In conclusion, the Massachusetts Enterprise Project generates net benefits from every 
perspective examined, including the government perspective. Given these strong results, the 
Massachusetts Enterprise Project appears to be a highly cost-effective policy tool for assisting 
UI claimants who are interested in pursuing self-employment. 

Washington 

The estimated benefits and costs in Washington are presented in Table 10.5. In 
Washington, participants experienced an increase of $1,093 in earnings during the 33-month 
observation period relative to the control group (see Table 10.1). As a resnlt of this inci-ease 
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Table 10.5 

Washington 
Benefit-Cost Analysis 
G)oUars per Claimant) 



Ben^ts and Costs 


:3Per8S>ective 


Partidpants 


Npjbipaiiidpaiits 


Society 


Earnings 


$1,093 


$0 


$1,093 


Taxes 


-164 


164 


0 


UI Payments 


1,013 


-1,013 


0 


Demonstration Costs 








Local Office Labor 


0 


-17 


-17 


Central Office Labor 


0 


-128 


-128 


Services 


0 


-252 


-252 


NET BENEFITS 


$1,942 


-$1,246 


$696 



in earnings, participants paid $164 in additional taxes."* In addition to their higher earnings, 
participants also received higher UI benefits (including the lump-sum payment) than their control 
group counterparts ($1,013). 

Summing all the gains and losses for participants we fmd that from the participants' 
perspective, the net benefit of SEED is $1 ,942 per participant. Summing all the gains and losses 
for nonparticipants, on the other hand, reveals that costs exceed benefits, yielding a net cost of - 
$1,246 per participant. Finally, summing the gains and losses for society as a whole yields a 
net benefit of $696 per participant. 

In Table 10.6, we present the benefits and costs from the government's perspective. As 
we did above for Massachusetts, we decompose total government into the Labor Departme ~t and 
all other government agencies. The results reported in Table 10.6 indicate that the Washington 
SEED program had a net cost to the Labor Department (-$1,399) and a net cost to the 
government as a whole (-$1,246). Other government agencies, however, experienced a net 
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"* Increased taxes on earnings are estimated as IS percent of earnings gains. 
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benefit of $153 from the additional taxes generated by the increased earnings of treatment group 
members relative to control group members. 

In conclusion, the Washington SEED program had positive net benefits from the 
participants' perspective and from society's perspective; it had net costs from the 
nonparticipants' perspective and from government's perspective. Thus, the Washington 

Table 10.6 

Washington 
Benefit-Cost Analysis 



Government Perspective 
(Dollars per Claimant) 



Benefits and Costs 


Perspective 


Labor 
Department 


Other 
Government 


Total 
Government 


Earnings 


$0 


$0 


$0 


Taxes 


+ 11 


+ 153 


+ 164 


UI Payments 


-1,013 


0 


-1,013 


Demonstration Costs 








Local Office Labor 


-17 


0 


-17 


Central Office Labor 


-128 


0 


-128 


Services 


-252 


0 


-252 


NET BENEFITS 


-$1,399 


$153 


-$1,246 



program, like the Massachusetts program, is cost-effective from society 's perspective. However, 
in contrast to the Massachusetts program, the Washington program has significant 
redistributional implications. Approximately two-thirds of the net benefits enjoyed by 
participants reflect direct costs to nonparticipants. 
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Sensitivity of the Net Benefit Estimates 

The cost-benefit results presented in previous sections are based on benefit and cost 
estimates that are subject to some uncertainty. In addition, a number of assumptions were 
necessary to derive the program impact estimates used in these cost-benefit calculations. For 
example, in estimating program impacts in Washington, we made a decision to include an outlier 
who dramatically affected the impact results. Moreover, in Massachusetts, we did not use the 
Cohort 3 results in the main impact analysis. Below we examine the sensitivity of the benefit- 
cost results to alternative assumptions on these issues. 

Excluding an Outlier in Washington 

As indicated in Chapter 7, the main SEED impact results included one treatment group 
member who reported self-employment earnings that were dramatically higher than the earnings 
of all other sample members (more than $500,000 over the Wave I observation period). Because 
a comparison of the survey data with administrative records available in Washington State could 
not convincingly demonstrate the survey data were incorrect, we decided to include the claimant 
in the impact analysis. We noted, however, that the annualized program impact estimates are 
reduced by some $400-$500 when this claimant is excluded. 

To determine the sensitivity of the overall benefit-cost results to the inclusion of this 
case, we excluded this case and reestimated the impacts on total earnings from wage and salary 
employment and self-employment over the 33 month observation period."' After excluding 
this case, the estimated impact of SEED on total earnings over the observation period is -$68, 
which is not statistically significantly different from zero. As shown in table 10.7, the benefit- 
cost results change dramatically when this claimant is excluded from the analysis. Specifically, 
the estimated benefits from the participants' perspective are reduced ^rom $1,942 to $955; the 
costs from the perspective of nonparticipants increase from -$1,246 to -$1,420; and the overall 



"' An alternative approach would have been to include the clainiant, but at some intermediate earnings value. 
However, the ESD data were insufficient to allow us to develop an alternative estimate of self-employment earnings 
for this case. 
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Table 10.7 

Washington 
Benefit-Cost Analysis 

Excluding Outlier 
(DoUars per Claimant) 



Benefits and Costs 


■ Perspective 


Partidpaiats { 


Nonpartidpants 


Society 


Earnings 


-$68 


$0 


-$68 


Taxes 


-f-10 


-10 


0 


UI Payments 


1,013 


-1,013 


0 


Demonstration Costs 








Local OfHce Labor 


0 


-17 


-17 


Central Office Labor 


0 


-128 


-128 


Services 


0 


-252 


-252 


NET BENEFITS 


$955 


-$1,420 


-$465 



net benefits per participant from society's perspective change from $696 to -$465. We view this 
as the lower bound estimate for SEED. 

These results indicate that excluding this claimant alters the conclusions on the cost- 
effectiveness of SEED. Without this individual, the Washington SEED program can no longer 
be viewed as a cost-effective program from society's perspective overall.'^" For SEED to be 
viewed as cost-effective from society's perspective, the monetary value of the other benefits that 
are not measured (e.g., improved psychological well-being, community economic development) 
would have to exceed $465 per participant. 
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'» It is interesting to note thtt excluding this case - and using an estimated impact on total earnings of -$68 - 
is very similar to using an impact estimate of $0, which is the value that would be used under the alternative 
procedure of using 0 when the estimated iaptcta are not significantly different from zero. 
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Using Cohort 3 Impact Estimates in Massachusetts 

As described in Chapter 4, a number of important changes occurred in the UI system at 
the time that Cohort 3 was being enrolled in the Massachusetts Enterprise Project. To assess 
the sensitivity of the Massachusetts cost-benefit results to changes in operating environment, we 
use the impact estimates derived from Cohort 3 in the cost-benefit analysis. As discussed in 
Chapter 8, we estimated program impacts separately for those who filed a UI claim before June 
1 , 1992 and those who filed after that date. In Tables 10.8 and 10.9, we present the benefit-cost 
results using the impact estimates from the two Cohort 3 subgroups.'^' 

As indicated in Table 10.8, the impact of the Massachusetts Enterprise Project on total 
earnings for this group was negative (-$540). Moreover, participants experienced a negligible 
reduction in UI payments. Summing the benefits and costs yields a net cost of $465 from the 
perspective of participants. Furthermore, as indicated in Table 10.8, there were also net costs 
from the perspective of nonparticipants and society as a whole. 
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Given the relatively short observation period available for Cohort 3 members, we use the annualized earning 
impacts. We also use the UI impacts in the first benefit year in these cost-benefit calculations. 
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Table 10.8 

Massachusetts 
Benefit-Cost Analysis 

Cohort 3: Prior to June 1, 1992 
(Dollars per Claunant) 



Benefits and Costs 


Perspective 


Participants 


Nonparticipants 


Society 


Earmngs 


-$540 


$0 


-$540 


Taxes 


+81 


-81 


0 


UI Payments 


-6 


-1-6 


0 


Demonstration Costs 








Local Office Labor 


0 


-56 


-56 


Central Office Labor 


0 


-178 


-178 


Services 


0 


-782 


-782 


NET BENEFITS 


-$465 


-$1,091 


-$1,556 
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Table 10.9 

Massachusetts 
BeneHt-Cost Analysis 

Cohort 3: After June 1, 1992 
(Dollars per Claimant) 



Benefits and Costs 


Perspective 


Participants 


Nonpartidpants 


Society 


Earnings 


+$1,371 


$0 


+$1,371 


Taxes 


-206 


+206 


0 


UI Payments 


+388 


-388 


0 


Demonstration Costs 








Local Office Labor 


0 


-56 


-56 


Central Office Labor 


0 


-178 


-178 


Services 


0 


-782 


-782 


NET BENEFITS 


+$1,553 


-$810 


+$355 
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Definition of Independent Variables Included in R^ression Models: 
Washington SEED Demonstration 


Independent Variables 




Treatment Group Dummy 


1 if claimant assigned to treatment group; 0 otherwise 


Vancouver Dummy 


1 if UI claim filed in Vancouver; 0 otherwise 


King County Dummy 


1 if UI claim filed in King County; 0 otherwise 


Snohomish County Dummy 


1 if UI claim filed in Snohomish County; 0 otherwise 


Wenatchee Dummy 


1 if UI claim filed in Wenatchee; 0 otherwise 


Yakima Dummy 


1 if UI claim filed in Yakima; 0 otherwise 


uenetit lear started m i^arter i ot lyyu 


1 if claimant's benefit year started in Quarter 1 of 1990; 0 
otherwise 


Benefit Year Started in Quarter 2 of 1990 


1 if claimant's benefit year started in Quarter 2 of 1990; 0 
otherwise 


Benefit Year Started in Quarter 4 of 1989 


1 if claimant's benefit year started in Quarter 4 of 1990; 0 
otherwise 


Age (in years) 


age in years 


Age Squared 


age^ 


Male Dummy 


1 if male; 0 otherwise 


White Dummy 


1 if white; 0 otherwise 


Completed College Dummy 


1 if highest grade completed is 16 or more; 0 otherwise 
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Deflnition of Independent Variables Included in Regression Models: 
Washington SEED Demonstration 


Independent Variables 




Prior Job in Professional, Technical or 
Managerial Profession Dummy 


1 if prior job in profession; 0 otherwise 


Prior Job in Services Sector Dummy 


1 if prior job was in Service Sector; 0 otherwise 


Intended to return to work to prior employer 


1 if claimant indicated on SEED application she/he intended to 
return to work to prior employer; 0 otherwise 


Spouse Employed Dummy 


1 if spouse employed; 0 otherwise 


Children under age six Dummy 


1 if claimant has children under the age of six; 0 otherwise 


Prior work experience in the area of the 
proposed business dummy 


1 if r»lQimont V»QC r\rinr Twnrlr PvnpriPnr'P in tllP JITPJI OT tllP 
1 11 ClalulalU iloi priur wUiK CApciiciivc ill ulc oica \ji uic 

proposed business; 0 otherwise 


Had a busmess at the time ot j)tbu 
application dummy 


1 if r'lQiTttont rpnnrt'^H hnvina n hll<linp<!<l Jit thp timp of SPPD 
1 11 L-lalindill FCUUriCU llaVlllt a UUolllCaa ai U1& lllllC \Ji 

application; 0 otherwise 


Self-employed prior to (but not at the time) 
of SEED application dummy 


1 11 Claimant was seii-cmpioyeu pnur lu occu appii^aiiun uui 
not at time of SEED application; 0 otherwise 


High wage earner in the four complete 
quarters before filing the UI claim dummy 


1 if claimant reported being in upper earnings quartile in the four 
complete quarters before filing UI claim; 0 otherwise 


Medium wage earner in the four complete 
quarters before filing the UI claim dummy 


1 if claimant reported being in two middle quartiles in the four 
complete quarters before filing UI claim; 0 otherwise 


Unemployment Rate in the claimant's county 
of residence during 1990 


Unemployment Rate in the claimant's county of residence during 
1990 
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Deflnition of Independent Variables Included in Regression Models: 
Massachusetts Enterprise Demonstration 


Independent Variables 




Treatment GrouD Dummv 


1 if claimant assigned to treatment group; 0 otherwise 


Gloucester Dummy (Cohort 1 only) 


1 if UI claim filed in Gloucester; 0 otherwise 


Greenfield Dummy 


1 if UI claim filed in Greenfield; 0 otherwise 


Lowell Dummy 


1 if UI claim filed in Lowell; 0 otherwise 


Milford Dummy (Cohorts 2 and 3 only) 


1 if UI chim filed in Milford; 0 otherwise 


New Bedford Dummy (Cohorts 1 and 2 only) 


1 if UI claim filed in New Bedford; 0 otherwise 


Northampton Dummy (Cohort 3 only) 


1 if UI claim filed in Northampton; 0 otherwise 


Roxbury County Dummy 


1 if UI claim filed in Roxbury County; 0 otherwise 


Springfield Dummy 


1 if UI claim filed in Springfield; 0 otherwise 


Woburn Dummy 


1 if UI claim filed in Woburn; 0 otherwise 


Benefit Year Started in Quarter 1 of 1990 


1 if claimant's benefit year started in Quarter 1 of 1990; 0 
otherwise 


Benefit Year Started in Quarter 2 of 1990 


1 if claimant's benefit year started in Quarter 2 of 1990; 0 
otherwise 


Benefit Year Started in Quarter 4 of 1989 


1 if claimant's benefit year started in Quarter 4 of 1990; 0 
otherwise 


Age (in years) 


age in years 


Age Squared 


age^ 


Male Dummy 


1 if male; 0 otherwise 


White Dummy 


1 if white; 0 otherwise 


Completed College Dummy 


1 if highest grade completed is 16 or more; 0 otherwise 
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Deflnition of Independent Variables Included in Regression Models: 
Massachusetts Enterprise Demonstration 


Independent Variable 




Prior Job in Professional, Technical or 
Managerial Profession Dummy 


1 if prior job in profession; 0 otherwise 


Prior Job in Services Sector Dummy 


1 if prior job was in Service Sector; 0 otherwise 


Intended to return to work to prior employer 


1 if claimant indicated on SEED application she/he intended to 
return to work to prior employer; 0 otherwise 


Children under age six Dunrniy 


1 if claimant has children under the age of six; 0 otherwise 


Prior work experience in the area of the 
proposed business dummy 


1 if claimant has prior work experience in the area of the 
proposed business; 0 otherwise 


mgu Wage Caiiici iii uic ii^ui x^yjiiiyi^i,^ 

quarters before filing the UI claim dummy 


1 if rlaimant renorterf beinp in unner earnings Quartile in the 
four complete quarters before filing UI claim; 0 otherwise 


Medium wage earner in the four complete 
quarters before fil-ng the UI claim dunrniy 


1 if claimant reported being in two middle quartiles in the four 
complete quarters before filing UI claim; 0 otherwise 


Unemployment Rate in the claimant's local 
office catchment area during 1990, 1991, 
1992, and 1993 


Unemployment Rate in the claimant's local office catchment 
area in 1990, 1991, 1992, and 1993 
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Notes for Appendix B 



In this appendix tlie definition of the column headings A, B, C and D vary according to whether the table 
describes self-employment outcomes, wage and salary outcomes, or combined self-employment and wage 

and salary employment outcomes. 

Self-Employment Outcomes 

A = whether the person experienced at least one spell of self-employment between random assignment and 

the date of the follow-up survey. 

B =annualized time spent in self-employment, between random assignment and the date of the follow-up 

survey. 

C =annualized real earnings (in 1990 dollars) from self-employment, between random assignment and the 

date of the follow-up survey. 

D =whether the person was in self-employment at the time of the follow-up survey. 

Wage and Salary Employment Outcomes 

A = whether the person experienced at least one spell of wage and salary employment between random 

assignment and the date of the follow-up survey. 

B = annualized time spent in wage and salary employment, between random assignment and the date of the 

follow-up survey. 

C = annualized real earnings (in 1990 dollars) from wage and salary employment, between random 

assignment and the date of the follow-up survey. 

D =whether the person was in wage and salary employment at the time of the follow-up survey. 
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Combined Self-Employment and Wage and Salary Employment Outcomes 

A = whether the person experienced at least one spell of either self-employment or wage and salary 
employment between random assignment and the date of the follow-up survey. 

B = annualized time spent in any type of employment, regardless of whether it was self-employment or 
wage and salary employment, between random assignment and the date of the follow-up survey. 

C = annualized real earnings (in 1990 dollars) from any type of employment, between random assignment 

and the date of the follow-up survey. 

D 7'v/hether the person was in either self -employment or wage and salary employment at the time of the 

' follow-up survey. 
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Washington Wave n 





Sdf'EmpIoystent | 


Wage and 
Salary J(A 


Combined SE 
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Massachusetts Cohorts 1 and 2, Wave I 
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Massachusetts Cohorts 1 and 2, Wave n 
Significant Coefficients 
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Effects on Subgroups 
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Notes for Appendix C 



This appendix defines the column headings L, B, C and D which appear in some of the impacts tables. 
The definition for each letter varies according to whether the table describes self-employment outcomes, 
wage and salary outcomes, or combined self-employment and wage and salary employment outcomes. 



Self-Employment Outcomes 

A = whether the person experienced at least one spell of self-employment between random assignment and 

the date of the follow-up survey. 

B =annualized time spent in self-employment, between random assignment and the date of the follow-up 

survey. 

C =annualized real earnings (in 1990 dollars) from self-employment, between random assignment and the 

date of the follow-up survey. 

D =whether the person was in self-employment at the time of the follow-up survey. 



Wage and Salary Employment Outcomes 

A = whether the person experienced at least one spell of wage and salary employment between random 

assignment and the date of the follow-up survey. 

B = annualized time spent in wage and salary employment, between random assignment and the date of the 

follow-up survey. 

C =annualized real earnings (in 1990 dollars) from wage and salary employment, between random 

assignment and the date of the follow-up survey. 

D = whether the person was in wage and salary employment at the time of the follow-up survey. 



Combined Self-Employment and Wage and Salary Employment Outcomes 

A = whether the person experienced at least one spell of either self-employment or wage and salary 
employment between random assignment and the date of the follow-up survey. 

B =annualized time spent in any type of employment, regardless of whether it was self-employment or 
wage and salary employment, between random assignment and the date of the follow-up survey. 

C =annualized real earnings (in 1990 dollars) from any type of employment, between random assignment 

and the date of the follow-up survey. 
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D = whether the person was in either self-employment or wage and salary employment at the time of the 

follow-up survey. 
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Subgroup Effects 
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Subgroup Effects 
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research findings and analyses dealing with unemployment 
insurance issues. Papers are prepared by research contractors, 
staff members of the unemployment insurance system, or individual 
researchers. Manuscripts and comments from interested 
individuals are welcomed. All correspondence should be sent to: 

UI Occasional Paper Series 
U.S. Department of Labor 
Unemployment Insurance Service 
Employment and Training Administration 
200 Constitution Ave., NW Room S-4519 
Washington, D.C. 20210 
Attention: Rosalind Thomas 

Arrangements have been made for the sale of most of the reports 
in the series through a Federal information and retrieval system, 
the National Technical Information Service (NTIS) . Copies of the 
reports are available from NTIS in paper or microfiche. The NTIS 
accession number for the paper copy is listed after the title of 
each paper. To obtain the papers from NTIS, forward orders to: 

National Technical Information Service 
U.S. Department of Commerce 
5285 Port Royal Road 
Springfield, Virginia 22161 
Telephone: (703) 487-4600 

Papers which are not available are indicated with an asterisk. 
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